Documentation: remove duplicated words

Remove many duplicated words under Documentation/ and do other small cleanups. Examples: "and and" --> "and" "in in" --> "in" "the the" --> "the" "the the" --> "to the" ... Signed-off-by: Paolo Ornati <ornati@fastwebnet.it> Signed-off-by: Adrian Bunk <bunk@stusta.de>

Documentation: remove duplicated words
Remove many duplicated words under Documentation/ and do other small cleanups. Examples: "and and" --> "and" "in in" --> "in" "the the" --> "the" "the the" --> "to the" ... Signed-off-by: Paolo Ornati <ornati@fastwebnet.it> Signed-off-by: Adrian Bunk <bunk@stusta.de>
Paolo Ornati · Adrian Bunk
1 parent 53cb47268e
Showing 52 changed files with 61 additions and 62 deletions Inline Diff
Documentation/DMA-mapping.txt
Documentation/DocBook/libata.tmpl
Documentation/DocBook/usb.tmpl
Documentation/RCU/whatisRCU.txt
Documentation/block/biodoc.txt
Documentation/driver-model/overview.txt
Documentation/exception.txt
Documentation/fb/fbcon.txt
Documentation/filesystems/directory-locking
Documentation/filesystems/files.txt
Documentation/filesystems/spufs.txt
Documentation/filesystems/tmpfs.txt
Documentation/filesystems/vfat.txt
Documentation/filesystems/vfs.txt
Documentation/fujitsu/frv/mmu-layout.txt
Documentation/ia64/efirtc.txt
Documentation/ia64/mca.txt
Documentation/input/input.txt
Documentation/isdn/INTERFACE.fax
Documentation/isdn/README.hysdn
 			Dynamic DMA mapping
 			===================
 		 David S. Miller <davem@redhat.com>
 		 Richard Henderson <rth@cygnus.com>
 		  Jakub Jelinek <jakub@redhat.com>
 This document describes the DMA mapping system in terms of the pci_
 API.  For a similar API that works for generic devices, see
 DMA-API.txt.
 Most of the 64bit platforms have special hardware that translates bus
 addresses (DMA addresses) into physical addresses.  This is similar to
 how page tables and/or a TLB translates virtual addresses to physical
 addresses on a CPU.  This is needed so that e.g. PCI devices can
 access with a Single Address Cycle (32bit DMA address) any page in the
 64bit physical address space.  Previously in Linux those 64bit
 platforms had to set artificial limits on the maximum RAM size in the
 system, so that the virt_to_bus() static scheme works (the DMA address
 translation tables were simply filled on bootup to map each bus
 address to the physical page __pa(bus_to_virt())).
 So that Linux can use the dynamic DMA mapping, it needs some help from the
 drivers, namely it has to take into account that DMA addresses should be
 mapped only for the time they are actually used and unmapped after the DMA
 transfer.
 The following API will work of course even on platforms where no such
 hardware exists, see e.g. include/asm-i386/pci.h for how it is implemented on
 top of the virt_to_bus interface.
 First of all, you should make sure
 #include <linux/pci.h>
 is in your driver. This file will obtain for you the definition of the
 dma_addr_t (which can hold any valid DMA address for the platform)
 type which should be used everywhere you hold a DMA (bus) address
 returned from the DMA mapping functions.
 			 What memory is DMA'able?
 The first piece of information you must know is what kernel memory can
 be used with the DMA mapping facilities.  There has been an unwritten
 set of rules regarding this, and this text is an attempt to finally
 write them down.
 If you acquired your memory via the page allocator
 (i.e. __get_free_page*()) or the generic memory allocators
 (i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from
 that memory using the addresses returned from those routines.
 This means specifically that you may _not_ use the memory/addresses
 returned from vmalloc() for DMA.  It is possible to DMA to the
 _underlying_ memory mapped into a vmalloc() area, but this requires
 walking page tables to get the physical addresses, and then
 translating each of those pages back to a kernel address using
 something like __va().  [ EDIT: Update this when we integrate
 Gerd Knorr's generic code which does this. ]
 This rule also means that you may use neither kernel image addresses
 (items in data/text/bss segments), nor module image addresses, nor
 stack addresses for DMA.  These could all be mapped somewhere entirely
 different than the rest of physical memory.  Even if those classes of
 memory could physically work with DMA, you'd need to ensure the I/O
 buffers were cacheline-aligned.  Without that, you'd see cacheline
 sharing problems (data corruption) on CPUs with DMA-incoherent caches.
 (The CPU could write to one word, DMA would write to a different one
 in the same cache line, and one of them could be overwritten.)
 Also, this means that you cannot take the return of a kmap()
 call and DMA to/from that.  This is similar to vmalloc().
 What about block I/O and networking buffers?  The block I/O and
 networking subsystems make sure that the buffers they use are valid
 for you to DMA from/to.
 			DMA addressing limitations
 Does your device have any DMA addressing limitations?  For example, is
 your device only capable of driving the low order 24-bits of address
 on the PCI bus for SAC DMA transfers?  If so, you need to inform the
 PCI layer of this fact.
 By default, the kernel assumes that your device can address the full
 32-bits in a SAC cycle.  For a 64-bit DAC capable device, this needs
 to be increased.  And for a device with limitations, as discussed in
 the previous paragraph, it needs to be decreased.
 pci_alloc_consistent() by default will return 32-bit DMA addresses.
 PCI-X specification requires PCI-X devices to support 64-bit
 addressing (DAC) for all transactions. And at least one platform (SGI
 SN2) requires 64-bit consistent allocations to operate correctly when
 the IO bus is in PCI-X mode. Therefore, like with pci_set_dma_mask(),
 it's good practice to call pci_set_consistent_dma_mask() to set the
 appropriate mask even if your device only supports 32-bit DMA
 (default) and especially if it's a PCI-X device.
 For correct operation, you must interrogate the PCI layer in your
 device probe routine to see if the PCI controller on the machine can
 properly support the DMA addressing limitation your device has.  It is
 good style to do this even if your device holds the default setting,
 because this shows that you did think about these issues wrt. your
 device.
 The query is performed via a call to pci_set_dma_mask():
 	int pci_set_dma_mask(struct pci_dev *pdev, u64 device_mask);
-The query for consistent allocations is performed via a a call to
+The query for consistent allocations is performed via a call to
 pci_set_consistent_dma_mask():
 	int pci_set_consistent_dma_mask(struct pci_dev *pdev, u64 device_mask);
 Here, pdev is a pointer to the PCI device struct of your device, and
 device_mask is a bit mask describing which bits of a PCI address your
 device supports.  It returns zero if your card can perform DMA
 properly on the machine given the address mask you provided.
 If it returns non-zero, your device cannot perform DMA properly on
 this platform, and attempting to do so will result in undefined
 behavior.  You must either use a different mask, or not use DMA.
 This means that in the failure case, you have three options:
 1) Use another DMA mask, if possible (see below).
 2) Use some non-DMA mode for data transfer, if possible.
 3) Ignore this device and do not initialize it.
 It is recommended that your driver print a kernel KERN_WARNING message
 when you end up performing either #2 or #3.  In this manner, if a user
 of your driver reports that performance is bad or that the device is not
 even detected, you can ask them for the kernel messages to find out
 exactly why.
 The standard 32-bit addressing PCI device would do something like
 this:
 	if (pci_set_dma_mask(pdev, DMA_32BIT_MASK)) {
 		printk(KERN_WARNING
 		       "mydev: No suitable DMA available.\n");
 		goto ignore_this_device;
 	}
 Another common scenario is a 64-bit capable device.  The approach
 here is to try for 64-bit DAC addressing, but back down to a
 32-bit mask should that fail.  The PCI platform code may fail the
 64-bit mask not because the platform is not capable of 64-bit
 addressing.  Rather, it may fail in this case simply because
 32-bit SAC addressing is done more efficiently than DAC addressing.
 Sparc64 is one platform which behaves in this way.
 Here is how you would handle a 64-bit capable device which can drive
 all 64-bits when accessing streaming DMA:
 	int using_dac;
 	if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) {
 		using_dac = 1;
 	} else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) {
 		using_dac = 0;
 	} else {
 		printk(KERN_WARNING
 		       "mydev: No suitable DMA available.\n");
 		goto ignore_this_device;
 	}
 If a card is capable of using 64-bit consistent allocations as well,
 the case would look like this:
 	int using_dac, consistent_using_dac;
 	if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) {
 		using_dac = 1;
 	   	consistent_using_dac = 1;
 		pci_set_consistent_dma_mask(pdev, DMA_64BIT_MASK);
 	} else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) {
 		using_dac = 0;
 		consistent_using_dac = 0;
 		pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);
 	} else {
 		printk(KERN_WARNING
 		       "mydev: No suitable DMA available.\n");
 		goto ignore_this_device;
 	}
 pci_set_consistent_dma_mask() will always be able to set the same or a
 smaller mask as pci_set_dma_mask(). However for the rare case that a
 device driver only uses consistent allocations, one would have to
 check the return value from pci_set_consistent_dma_mask().
 If your 64-bit device is going to be an enormous consumer of DMA
 mappings, this can be problematic since the DMA mappings are a
 finite resource on many platforms.  Please see the "DAC Addressing
 for Address Space Hungry Devices" section near the end of this
 document for how to handle this case.
 Finally, if your device can only drive the low 24-bits of
 address during PCI bus mastering you might do something like:
 	if (pci_set_dma_mask(pdev, DMA_24BIT_MASK)) {
 		printk(KERN_WARNING
 		       "mydev: 24-bit DMA addressing not available.\n");
 		goto ignore_this_device;
 	}
 [Better use DMA_24BIT_MASK instead of 0x00ffffff.
 See linux/include/dma-mapping.h for reference.]
 When pci_set_dma_mask() is successful, and returns zero, the PCI layer
 saves away this mask you have provided.  The PCI layer will use this
 information later when you make DMA mappings.
 There is a case which we are aware of at this time, which is worth
 mentioning in this documentation.  If your device supports multiple
 functions (for example a sound card provides playback and record
 functions) and the various different functions have _different_
 DMA addressing limitations, you may wish to probe each mask and
 only provide the functionality which the machine can handle.  It
 is important that the last call to pci_set_dma_mask() be for the
 most specific mask.
 Here is pseudo-code showing how this might be done:
 	#define PLAYBACK_ADDRESS_BITS	DMA_32BIT_MASK
 	#define RECORD_ADDRESS_BITS	0x00ffffff
 	struct my_sound_card *card;
 	struct pci_dev *pdev;
 	...
 	if (!pci_set_dma_mask(pdev, PLAYBACK_ADDRESS_BITS)) {
 		card->playback_enabled = 1;
 	} else {
 		card->playback_enabled = 0;
 		printk(KERN_WARN "%s: Playback disabled due to DMA limitations.\n",
 		       card->name);
 	}
 	if (!pci_set_dma_mask(pdev, RECORD_ADDRESS_BITS)) {
 		card->record_enabled = 1;
 	} else {
 		card->record_enabled = 0;
 		printk(KERN_WARN "%s: Record disabled due to DMA limitations.\n",
 		       card->name);
 	}
 A sound card was used as an example here because this genre of PCI
 devices seems to be littered with ISA chips given a PCI front end,
 and thus retaining the 16MB DMA addressing limitations of ISA.
 			Types of DMA mappings
 There are two types of DMA mappings:
 - Consistent DMA mappings which are usually mapped at driver
  initialization, unmapped at the end and for which the hardware should
  guarantee that the device and the CPU can access the data
  in parallel and will see updates made by each other without any
  explicit software flushing.
  Think of "consistent" as "synchronous" or "coherent".
  The current default is to return consistent memory in the low 32
  bits of the PCI bus space.  However, for future compatibility you
  should set the consistent mask even if this default is fine for your
  driver.
  Good examples of what to use consistent mappings for are:
 	- Network card DMA ring descriptors.
 	- SCSI adapter mailbox command data structures.
 	- Device firmware microcode executed out of
 	  main memory.
  The invariant these examples all require is that any CPU store
  to memory is immediately visible to the device, and vice
  versa.  Consistent mappings guarantee this.
  IMPORTANT: Consistent DMA memory does not preclude the usage of
             proper memory barriers.  The CPU may reorder stores to
 	     consistent memory just as it may normal memory.  Example:
 	     if it is important for the device to see the first word
 	     of a descriptor updated before the second, you must do
 	     something like:
 		desc->word0 = address;
 		wmb();
 		desc->word1 = DESC_VALID;
             in order to get correct behavior on all platforms.
 	     Also, on some platforms your driver may need to flush CPU write
 	     buffers in much the same way as it needs to flush write buffers
 	     found in PCI bridges (such as by reading a register's value
 	     after writing it).
 - Streaming DMA mappings which are usually mapped for one DMA transfer,
  unmapped right after it (unless you use pci_dma_sync_* below) and for which
  hardware can optimize for sequential accesses.
  This of "streaming" as "asynchronous" or "outside the coherency
  domain".
  Good examples of what to use streaming mappings for are:
 	- Networking buffers transmitted/received by a device.
 	- Filesystem buffers written/read by a SCSI device.
  The interfaces for using this type of mapping were designed in
  such a way that an implementation can make whatever performance
  optimizations the hardware allows.  To this end, when using
  such mappings you must be explicit about what you want to happen.
 Neither type of DMA mapping has alignment restrictions that come
 from PCI, although some devices may have such restrictions.
 Also, systems with caches that aren't DMA-coherent will work better
 when the underlying buffers don't share cache lines with other data.
 		 Using Consistent DMA mappings.
 To allocate and map large (PAGE_SIZE or so) consistent DMA regions,
 you should do:
 	dma_addr_t dma_handle;
 	cpu_addr = pci_alloc_consistent(dev, size, &dma_handle);
 where dev is a struct pci_dev *. You should pass NULL for PCI like buses
 where devices don't have struct pci_dev (like ISA, EISA).  This may be
 called in interrupt context. 
 This argument is needed because the DMA translations may be bus
 specific (and often is private to the bus which the device is attached
 to).
 Size is the length of the region you want to allocate, in bytes.
 This routine will allocate RAM for that region, so it acts similarly to
 __get_free_pages (but takes size instead of a page order).  If your
 driver needs regions sized smaller than a page, you may prefer using
 the pci_pool interface, described below.
 The consistent DMA mapping interfaces, for non-NULL dev, will by
 default return a DMA address which is SAC (Single Address Cycle)
 addressable.  Even if the device indicates (via PCI dma mask) that it
 may address the upper 32-bits and thus perform DAC cycles, consistent
 allocation will only return > 32-bit PCI addresses for DMA if the
 consistent dma mask has been explicitly changed via
 pci_set_consistent_dma_mask().  This is true of the pci_pool interface
 as well.
 pci_alloc_consistent returns two values: the virtual address which you
 can use to access it from the CPU and dma_handle which you pass to the
 card.
 The cpu return address and the DMA bus master address are both
 guaranteed to be aligned to the smallest PAGE_SIZE order which
 is greater than or equal to the requested size.  This invariant
 exists (for example) to guarantee that if you allocate a chunk
 which is smaller than or equal to 64 kilobytes, the extent of the
 buffer you receive will not cross a 64K boundary.
 To unmap and free such a DMA region, you call:
 	pci_free_consistent(dev, size, cpu_addr, dma_handle);
 where dev, size are the same as in the above call and cpu_addr and
 dma_handle are the values pci_alloc_consistent returned to you.
 This function may not be called in interrupt context.
 If your driver needs lots of smaller memory regions, you can write
 custom code to subdivide pages returned by pci_alloc_consistent,
 or you can use the pci_pool API to do that.  A pci_pool is like
 a kmem_cache, but it uses pci_alloc_consistent not __get_free_pages.
 Also, it understands common hardware constraints for alignment,
 like queue heads needing to be aligned on N byte boundaries.
 Create a pci_pool like this:
 	struct pci_pool *pool;
 	pool = pci_pool_create(name, dev, size, align, alloc);
 The "name" is for diagnostics (like a kmem_cache name); dev and size
 are as above.  The device's hardware alignment requirement for this
 type of data is "align" (which is expressed in bytes, and must be a
 power of two).  If your device has no boundary crossing restrictions,
 pass 0 for alloc; passing 4096 says memory allocated from this pool
 must not cross 4KByte boundaries (but at that time it may be better to
 go for pci_alloc_consistent directly instead).
 Allocate memory from a pci pool like this:
 	cpu_addr = pci_pool_alloc(pool, flags, &dma_handle);
 flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor
 holding SMP locks), SLAB_ATOMIC otherwise.  Like pci_alloc_consistent,
 this returns two values, cpu_addr and dma_handle.
 Free memory that was allocated from a pci_pool like this:
 	pci_pool_free(pool, cpu_addr, dma_handle);
 where pool is what you passed to pci_pool_alloc, and cpu_addr and
 dma_handle are the values pci_pool_alloc returned. This function
 may be called in interrupt context.
 Destroy a pci_pool by calling:
 	pci_pool_destroy(pool);
 Make sure you've called pci_pool_free for all memory allocated
 from a pool before you destroy the pool. This function may not
 be called in interrupt context.
 			DMA Direction
 The interfaces described in subsequent portions of this document
 take a DMA direction argument, which is an integer and takes on
 one of the following values:
 PCI_DMA_BIDIRECTIONAL
 PCI_DMA_TODEVICE
 PCI_DMA_FROMDEVICE
 PCI_DMA_NONE
 One should provide the exact DMA direction if you know it.
 PCI_DMA_TODEVICE means "from main memory to the PCI device"
 PCI_DMA_FROMDEVICE means "from the PCI device to main memory"
 It is the direction in which the data moves during the DMA
 transfer.
 You are _strongly_ encouraged to specify this as precisely
 as you possibly can.
 If you absolutely cannot know the direction of the DMA transfer,
 specify PCI_DMA_BIDIRECTIONAL.  It means that the DMA can go in
 either direction.  The platform guarantees that you may legally
 specify this, and that it will work, but this may be at the
 cost of performance for example.
 The value PCI_DMA_NONE is to be used for debugging.  One can
 hold this in a data structure before you come to know the
 precise direction, and this will help catch cases where your
 direction tracking logic has failed to set things up properly.
 Another advantage of specifying this value precisely (outside of
 potential platform-specific optimizations of such) is for debugging.
 Some platforms actually have a write permission boolean which DMA
 mappings can be marked with, much like page protections in the user
 program address space.  Such platforms can and do report errors in the
 kernel logs when the PCI controller hardware detects violation of the
 permission setting.
 Only streaming mappings specify a direction, consistent mappings
 implicitly have a direction attribute setting of
 PCI_DMA_BIDIRECTIONAL.
 The SCSI subsystem tells you the direction to use in the
 'sc_data_direction' member of the SCSI command your driver is
 working on.
 For Networking drivers, it's a rather simple affair.  For transmit
 packets, map/unmap them with the PCI_DMA_TODEVICE direction
 specifier.  For receive packets, just the opposite, map/unmap them
 with the PCI_DMA_FROMDEVICE direction specifier.
 		  Using Streaming DMA mappings
 The streaming DMA mapping routines can be called from interrupt
 context.  There are two versions of each map/unmap, one which will
 map/unmap a single memory region, and one which will map/unmap a
 scatterlist.
 To map a single region, you do:
 	struct pci_dev *pdev = mydev->pdev;
 	dma_addr_t dma_handle;
 	void *addr = buffer->ptr;
 	size_t size = buffer->len;
 	dma_handle = pci_map_single(dev, addr, size, direction);
 and to unmap it:
 	pci_unmap_single(dev, dma_handle, size, direction);
 You should call pci_unmap_single when the DMA activity is finished, e.g.
 from the interrupt which told you that the DMA transfer is done.
 Using cpu pointers like this for single mappings has a disadvantage,
 you cannot reference HIGHMEM memory in this way.  Thus, there is a
 map/unmap interface pair akin to pci_{map,unmap}_single.  These
 interfaces deal with page/offset pairs instead of cpu pointers.
 Specifically:
 	struct pci_dev *pdev = mydev->pdev;
 	dma_addr_t dma_handle;
 	struct page *page = buffer->page;
 	unsigned long offset = buffer->offset;
 	size_t size = buffer->len;
 	dma_handle = pci_map_page(dev, page, offset, size, direction);
 	...
 	pci_unmap_page(dev, dma_handle, size, direction);
 Here, "offset" means byte offset within the given page.
 With scatterlists, you map a region gathered from several regions by:
 	int i, count = pci_map_sg(dev, sglist, nents, direction);
 	struct scatterlist *sg;
 	for (i = 0, sg = sglist; i < count; i++, sg++) {
 		hw_address[i] = sg_dma_address(sg);
 		hw_len[i] = sg_dma_len(sg);
 	}
 where nents is the number of entries in the sglist.
 The implementation is free to merge several consecutive sglist entries
 into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any
 consecutive sglist entries can be merged into one provided the first one
 ends and the second one starts on a page boundary - in fact this is a huge
 advantage for cards which either cannot do scatter-gather or have very
 limited number of scatter-gather entries) and returns the actual number
 of sg entries it mapped them to. On failure 0 is returned.
 Then you should loop count times (note: this can be less than nents times)
 and use sg_dma_address() and sg_dma_len() macros where you previously
 accessed sg->address and sg->length as shown above.
 To unmap a scatterlist, just call:
 	pci_unmap_sg(dev, sglist, nents, direction);
 Again, make sure DMA activity has already finished.
 PLEASE NOTE:  The 'nents' argument to the pci_unmap_sg call must be
              the _same_ one you passed into the pci_map_sg call,
 	      it should _NOT_ be the 'count' value _returned_ from the
              pci_map_sg call.
 Every pci_map_{single,sg} call should have its pci_unmap_{single,sg}
 counterpart, because the bus address space is a shared resource (although
 in some ports the mapping is per each BUS so less devices contend for the
 same bus address space) and you could render the machine unusable by eating
 all bus addresses.
 If you need to use the same streaming DMA region multiple times and touch
 the data in between the DMA transfers, the buffer needs to be synced
 properly in order for the cpu and device to see the most uptodate and
 correct copy of the DMA buffer.
 So, firstly, just map it with pci_map_{single,sg}, and after each DMA
 transfer call either:
 	pci_dma_sync_single_for_cpu(dev, dma_handle, size, direction);
 or:
 	pci_dma_sync_sg_for_cpu(dev, sglist, nents, direction);
 as appropriate.
 Then, if you wish to let the device get at the DMA area again,
 finish accessing the data with the cpu, and then before actually
 giving the buffer to the hardware call either:
 	pci_dma_sync_single_for_device(dev, dma_handle, size, direction);
 or:
 	pci_dma_sync_sg_for_device(dev, sglist, nents, direction);
 as appropriate.
 After the last DMA transfer call one of the DMA unmap routines
 pci_unmap_{single,sg}. If you don't touch the data from the first pci_map_*
 call till pci_unmap_*, then you don't have to call the pci_dma_sync_*
 routines at all.
 Here is pseudo code which shows a situation in which you would need
 to use the pci_dma_sync_*() interfaces.
 	my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len)
 	{
 		dma_addr_t mapping;
 		mapping = pci_map_single(cp->pdev, buffer, len, PCI_DMA_FROMDEVICE);
 		cp->rx_buf = buffer;
 		cp->rx_len = len;
 		cp->rx_dma = mapping;
 		give_rx_buf_to_card(cp);
 	}
 	...
 	my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs)
 	{
 		struct my_card *cp = devid;
 		...
 		if (read_card_status(cp) == RX_BUF_TRANSFERRED) {
 			struct my_card_header *hp;
 			/* Examine the header to see if we wish
 			 * to accept the data.  But synchronize
 			 * the DMA transfer with the CPU first
 			 * so that we see updated contents.
 			 */
 			pci_dma_sync_single_for_cpu(cp->pdev, cp->rx_dma,
 						    cp->rx_len,
 						    PCI_DMA_FROMDEVICE);
 			/* Now it is safe to examine the buffer. */
 			hp = (struct my_card_header *) cp->rx_buf;
 			if (header_is_ok(hp)) {
 				pci_unmap_single(cp->pdev, cp->rx_dma, cp->rx_len,
 						 PCI_DMA_FROMDEVICE);
 				pass_to_upper_layers(cp->rx_buf);
 				make_and_setup_new_rx_buf(cp);
 			} else {
 				/* Just sync the buffer and give it back
 				 * to the card.
 				 */
 				pci_dma_sync_single_for_device(cp->pdev,
 							       cp->rx_dma,
 							       cp->rx_len,
 							       PCI_DMA_FROMDEVICE);
 				give_rx_buf_to_card(cp);
 			}
 		}
 	}
 Drivers converted fully to this interface should not use virt_to_bus any
 longer, nor should they use bus_to_virt. Some drivers have to be changed a
 little bit, because there is no longer an equivalent to bus_to_virt in the
 dynamic DMA mapping scheme - you have to always store the DMA addresses
 returned by the pci_alloc_consistent, pci_pool_alloc, and pci_map_single
 calls (pci_map_sg stores them in the scatterlist itself if the platform
 supports dynamic DMA mapping in hardware) in your driver structures and/or
 in the card registers.
 All PCI drivers should be using these interfaces with no exceptions.
 It is planned to completely remove virt_to_bus() and bus_to_virt() as
 they are entirely deprecated.  Some ports already do not provide these
 as it is impossible to correctly support them.
 		64-bit DMA and DAC cycle support
 Do you understand all of the text above?  Great, then you already
 know how to use 64-bit DMA addressing under Linux.  Simply make
 the appropriate pci_set_dma_mask() calls based upon your cards
 capabilities, then use the mapping APIs above.
 It is that simple.
 Well, not for some odd devices.  See the next section for information
 about that.
 	DAC Addressing for Address Space Hungry Devices
 There exists a class of devices which do not mesh well with the PCI
 DMA mapping API.  By definition these "mappings" are a finite
 resource.  The number of total available mappings per bus is platform
 specific, but there will always be a reasonable amount.
 What is "reasonable"?  Reasonable means that networking and block I/O
 devices need not worry about using too many mappings.
 As an example of a problematic device, consider compute cluster cards.
 They can potentially need to access gigabytes of memory at once via
 DMA.  Dynamic mappings are unsuitable for this kind of access pattern.
 To this end we've provided a small API by which a device driver
 may use DAC cycles to directly address all of physical memory.
 Not all platforms support this, but most do.  It is easy to determine
 whether the platform will work properly at probe time.
 First, understand that there may be a SEVERE performance penalty for
 using these interfaces on some platforms.  Therefore, you MUST only
 use these interfaces if it is absolutely required.  %99 of devices can
 use the normal APIs without any problems.
 Note that for streaming type mappings you must either use these
 interfaces, or the dynamic mapping interfaces above.  You may not mix
 usage of both for the same device.  Such an act is illegal and is
 guaranteed to put a banana in your tailpipe.
 However, consistent mappings may in fact be used in conjunction with
 these interfaces.  Remember that, as defined, consistent mappings are
 always going to be SAC addressable.
 The first thing your driver needs to do is query the PCI platform
 layer if it is capable of handling your devices DAC addressing
 capabilities:
 	int pci_dac_dma_supported(struct pci_dev *hwdev, u64 mask);
 You may not use the following interfaces if this routine fails.
 Next, DMA addresses using this API are kept track of using the
 dma64_addr_t type.  It is guaranteed to be big enough to hold any
 DAC address the platform layer will give to you from the following
 routines.  If you have consistent mappings as well, you still
 use plain dma_addr_t to keep track of those.
 All mappings obtained here will be direct.  The mappings are not
 translated, and this is the purpose of this dialect of the DMA API.
 All routines work with page/offset pairs.  This is the _ONLY_ way to 
 portably refer to any piece of memory.  If you have a cpu pointer
 (which may be validly DMA'd too) you may easily obtain the page
 and offset using something like this:
 	struct page *page = virt_to_page(ptr);
 	unsigned long offset = offset_in_page(ptr);
 Here are the interfaces:
 	dma64_addr_t pci_dac_page_to_dma(struct pci_dev *pdev,
 					 struct page *page,
 					 unsigned long offset,
 					 int direction);
 The DAC address for the tuple PAGE/OFFSET are returned.  The direction
 argument is the same as for pci_{map,unmap}_single().  The same rules
 for cpu/device access apply here as for the streaming mapping
 interfaces.  To reiterate:
 	The cpu may touch the buffer before pci_dac_page_to_dma.
 	The device may touch the buffer after pci_dac_page_to_dma
 	is made, but the cpu may NOT.
 When the DMA transfer is complete, invoke:
 	void pci_dac_dma_sync_single_for_cpu(struct pci_dev *pdev,
 					     dma64_addr_t dma_addr,
 					     size_t len, int direction);
 This must be done before the CPU looks at the buffer again.
 This interface behaves identically to pci_dma_sync_{single,sg}_for_cpu().
 And likewise, if you wish to let the device get back at the buffer after
 the cpu has read/written it, invoke:
 	void pci_dac_dma_sync_single_for_device(struct pci_dev *pdev,
 						dma64_addr_t dma_addr,
 						size_t len, int direction);
 before letting the device access the DMA area again.
 If you need to get back to the PAGE/OFFSET tuple from a dma64_addr_t
 the following interfaces are provided:
 	struct page *pci_dac_dma_to_page(struct pci_dev *pdev,
 					 dma64_addr_t dma_addr);
 	unsigned long pci_dac_dma_to_offset(struct pci_dev *pdev,
 					    dma64_addr_t dma_addr);
 This is possible with the DAC interfaces purely because they are
 not translated in any way.
 		Optimizing Unmap State Space Consumption
 On many platforms, pci_unmap_{single,page}() is simply a nop.
 Therefore, keeping track of the mapping address and length is a waste
 of space.  Instead of filling your drivers up with ifdefs and the like
 to "work around" this (which would defeat the whole purpose of a
 portable API) the following facilities are provided.
 Actually, instead of describing the macros one by one, we'll
 transform some example code.
 1) Use DECLARE_PCI_UNMAP_{ADDR,LEN} in state saving structures.
   Example, before:
 	struct ring_state {
 		struct sk_buff *skb;
 		dma_addr_t mapping;
 		__u32 len;
 	};
   after:
 	struct ring_state {
 		struct sk_buff *skb;
 		DECLARE_PCI_UNMAP_ADDR(mapping)
 		DECLARE_PCI_UNMAP_LEN(len)
 	};
   NOTE: DO NOT put a semicolon at the end of the DECLARE_*()
         macro.
 2) Use pci_unmap_{addr,len}_set to set these values.
   Example, before:
 	ringp->mapping = FOO;
 	ringp->len = BAR;
   after:
 	pci_unmap_addr_set(ringp, mapping, FOO);
 	pci_unmap_len_set(ringp, len, BAR);
 3) Use pci_unmap_{addr,len} to access these values.
   Example, before:
 	pci_unmap_single(pdev, ringp->mapping, ringp->len,
 			 PCI_DMA_FROMDEVICE);
   after:
 	pci_unmap_single(pdev,
 			 pci_unmap_addr(ringp, mapping),
 			 pci_unmap_len(ringp, len),
 			 PCI_DMA_FROMDEVICE);
 It really should be self-explanatory.  We treat the ADDR and LEN
 separately, because it is possible for an implementation to only
 need the address in order to perform the unmap operation.
 			Platform Issues
 If you are just writing drivers for Linux and do not maintain
 an architecture port for the kernel, you can safely skip down
 to "Closing".
 1) Struct scatterlist requirements.
   Struct scatterlist must contain, at a minimum, the following
   members:
 	struct page *page;
 	unsigned int offset;
 	unsigned int length;
   The base address is specified by a "page+offset" pair.
   Previous versions of struct scatterlist contained a "void *address"
   field that was sometimes used instead of page+offset.  As of Linux
   2.5., page+offset is always used, and the "address" field has been
   deleted.
 2) More to come...
 			Handling Errors
 DMA address space is limited on some architectures and an allocation
 failure can be determined by:
 - checking if pci_alloc_consistent returns NULL or pci_map_sg returns 0
 - checking the returned dma_addr_t of pci_map_single and pci_map_page
  by using pci_dma_mapping_error():
 	dma_addr_t dma_handle;
 	dma_handle = pci_map_single(dev, addr, size, direction);
 	if (pci_dma_mapping_error(dma_handle)) {
 		/*
 		 * reduce current DMA mapping usage,
 		 * delay and try again later or
 		 * reset driver.
 		 */
 	}
 			   Closing
 This document, and the API itself, would not be in it's current
 form without the feedback and suggestions from numerous individuals.
 We would like to specifically mention, in no particular order, the
 following people:
 	Russell King <rmk@arm.linux.org.uk>
 	Leo Dagum <dagum@barrel.engr.sgi.com>
 	Ralf Baechle <ralf@oss.sgi.com>
 	Grant Grundler <grundler@cup.hp.com>
 	Jay Estabrook <Jay.Estabrook@compaq.com>
 	Thomas Sailer <sailer@ife.ee.ethz.ch>
 	Andrea Arcangeli <andrea@suse.de>
 	Jens Axboe <axboe@suse.de>
 	David Mosberger-Tang <davidm@hpl.hp.com>
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
 	"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
 <book id="libataDevGuide">
 <bookinfo>
  <title>libATA Developer's Guide</title>
  <authorgroup>
   <author>
    <firstname>Jeff</firstname>
    <surname>Garzik</surname>
   </author>
  </authorgroup>
  <copyright>
   <year>2003-2005</year>
   <holder>Jeff Garzik</holder>
  </copyright>
  <legalnotice>
   <para>
   The contents of this file are subject to the Open
   Software License version 1.1 that can be found at
   <ulink url="http://www.opensource.org/licenses/osl-1.1.txt">http://www.opensource.org/licenses/osl-1.1.txt</ulink> and is included herein
   by reference.
   </para>
   <para>
   Alternatively, the contents of this file may be used under the terms
   of the GNU General Public License version 2 (the "GPL") as distributed
   in the kernel source COPYING file, in which case the provisions of
   the GPL are applicable instead of the above.  If you wish to allow
   the use of your version of this file only under the terms of the
   GPL and not to allow others to use your version of this file under
   the OSL, indicate your decision by deleting the provisions above and
   replace them with the notice and other provisions required by the GPL.
   If you do not delete the provisions above, a recipient may use your
   version of this file under either the OSL or the GPL.
   </para>
  </legalnotice>
 </bookinfo>
 <toc></toc>
  <chapter id="libataIntroduction">
     <title>Introduction</title>
  <para>
  libATA is a library used inside the Linux kernel to support ATA host
  controllers and devices.  libATA provides an ATA driver API, class
  transports for ATA and ATAPI devices, and SCSI&lt;-&gt;ATA translation
  for ATA devices according to the T10 SAT specification.
  </para>
  <para>
  This Guide documents the libATA driver API, library functions, library
  internals, and a couple sample ATA low-level drivers.
  </para>
  </chapter>
  <chapter id="libataDriverApi">
     <title>libata Driver API</title>
     <para>
     struct ata_port_operations is defined for every low-level libata
     hardware driver, and it controls how the low-level driver
     interfaces with the ATA and SCSI layers.
     </para>
     <para>
     FIS-based drivers will hook into the system with ->qc_prep() and
     ->qc_issue() high-level hooks.  Hardware which behaves in a manner
     similar to PCI IDE hardware may utilize several generic helpers,
     defining at a bare minimum the bus I/O addresses of the ATA shadow
     register blocks.
     </para>
     <sect1>
        <title>struct ata_port_operations</title>
 	<sect2><title>Disable ATA port</title>
 	<programlisting>
 void (*port_disable) (struct ata_port *);
 	</programlisting>
 	<para>
 	Called from ata_bus_probe() and ata_bus_reset() error paths,
 	as well as when unregistering from the SCSI module (rmmod, hot
 	unplug).
 	This function should do whatever needs to be done to take the
 	port out of use.  In most cases, ata_port_disable() can be used
 	as this hook.
 	</para>
 	<para>
 	Called from ata_bus_probe() on a failed probe.
 	Called from ata_bus_reset() on a failed bus reset.
 	Called from ata_scsi_release().
 	</para>
 	</sect2>
 	<sect2><title>Post-IDENTIFY device configuration</title>
 	<programlisting>
 void (*dev_config) (struct ata_port *, struct ata_device *);
 	</programlisting>
 	<para>
 	Called after IDENTIFY [PACKET] DEVICE is issued to each device
 	found.  Typically used to apply device-specific fixups prior to
 	issue of SET FEATURES - XFER MODE, and prior to operation.
 	</para>
 	<para>
 	Called by ata_device_add() after ata_dev_identify() determines
 	a device is present.
 	</para>
 	<para>
 	This entry may be specified as NULL in ata_port_operations.
 	</para>
 	</sect2>
 	<sect2><title>Set PIO/DMA mode</title>
 	<programlisting>
 void (*set_piomode) (struct ata_port *, struct ata_device *);
 void (*set_dmamode) (struct ata_port *, struct ata_device *);
 void (*post_set_mode) (struct ata_port *);
 unsigned int (*mode_filter) (struct ata_port *, struct ata_device *, unsigned int);
 	</programlisting>
 	<para>
 	Hooks called prior to the issue of SET FEATURES - XFER MODE
 	command.  The optional ->mode_filter() hook is called when libata
 	has built a mask of the possible modes. This is passed to the 
 	->mode_filter() function which should return a mask of valid modes
 	after filtering those unsuitable due to hardware limits. It is not
 	valid to use this interface to add modes.
 	</para>
 	<para>
 	dev->pio_mode and dev->dma_mode are guaranteed to be valid when
 	->set_piomode() and when ->set_dmamode() is called. The timings for
 	any other drive sharing the cable will also be valid at this point.
 	That is the library records the decisions for the modes of each
 	drive on a channel before it attempts to set any of them.
 	</para>
 	<para>
 	->post_set_mode() is
 	called unconditionally, after the SET FEATURES - XFER MODE
 	command completes successfully.
 	</para>
 	<para>
 	->set_piomode() is always called (if present), but
 	->set_dma_mode() is only called if DMA is possible.
 	</para>
 	</sect2>
 	<sect2><title>Taskfile read/write</title>
 	<programlisting>
 void (*tf_load) (struct ata_port *ap, struct ata_taskfile *tf);
 void (*tf_read) (struct ata_port *ap, struct ata_taskfile *tf);
 	</programlisting>
 	<para>
 	->tf_load() is called to load the given taskfile into hardware
 	registers / DMA buffers.  ->tf_read() is called to read the
 	hardware registers / DMA buffers, to obtain the current set of
 	taskfile register values.
 	Most drivers for taskfile-based hardware (PIO or MMIO) use
 	ata_tf_load() and ata_tf_read() for these hooks.
 	</para>
 	</sect2>
 	<sect2><title>PIO data read/write</title>
 	<programlisting>
 void (*data_xfer) (struct ata_device *, unsigned char *, unsigned int, int);
 	</programlisting>
 	<para>
 All bmdma-style drivers must implement this hook.  This is the low-level
 operation that actually copies the data bytes during a PIO data
 transfer.
 Typically the driver
 will choose one of ata_pio_data_xfer_noirq(), ata_pio_data_xfer(), or
 ata_mmio_data_xfer().
 	</para>
 	</sect2>
 	<sect2><title>ATA command execute</title>
 	<programlisting>
 void (*exec_command)(struct ata_port *ap, struct ata_taskfile *tf);
 	</programlisting>
 	<para>
 	causes an ATA command, previously loaded with
 	->tf_load(), to be initiated in hardware.
 	Most drivers for taskfile-based hardware use ata_exec_command()
 	for this hook.
 	</para>
 	</sect2>
 	<sect2><title>Per-cmd ATAPI DMA capabilities filter</title>
 	<programlisting>
 int (*check_atapi_dma) (struct ata_queued_cmd *qc);
 	</programlisting>
 	<para>
 Allow low-level driver to filter ATA PACKET commands, returning a status
 indicating whether or not it is OK to use DMA for the supplied PACKET
 command.
 	</para>
 	<para>
 	This hook may be specified as NULL, in which case libata will
 	assume that atapi dma can be supported.
 	</para>
 	</sect2>
 	<sect2><title>Read specific ATA shadow registers</title>
 	<programlisting>
 u8   (*check_status)(struct ata_port *ap);
 u8   (*check_altstatus)(struct ata_port *ap);
 	</programlisting>
 	<para>
 	Reads the Status/AltStatus ATA shadow register from
 	hardware.  On some hardware, reading the Status register has
 	the side effect of clearing the interrupt condition.
 	Most drivers for taskfile-based hardware use
 	ata_check_status() for this hook.
 	</para>
 	<para>
 	Note that because this is called from ata_device_add(), at
 	least a dummy function that clears device interrupts must be
 	provided for all drivers, even if the controller doesn't
 	actually have a taskfile status register.
 	</para>
 	</sect2>
 	<sect2><title>Select ATA device on bus</title>
 	<programlisting>
 void (*dev_select)(struct ata_port *ap, unsigned int device);
 	</programlisting>
 	<para>
 	Issues the low-level hardware command(s) that causes one of N
 	hardware devices to be considered 'selected' (active and
 	available for use) on the ATA bus.  This generally has no
 	meaning on FIS-based devices.
 	</para>
 	<para>
 	Most drivers for taskfile-based hardware use
 	ata_std_dev_select() for this hook.  Controllers which do not
 	support second drives on a port (such as SATA contollers) will
 	use ata_noop_dev_select().
 	</para>
 	</sect2>
 	<sect2><title>Private tuning method</title>
 	<programlisting>
 void (*set_mode) (struct ata_port *ap);
 	</programlisting>
 	<para>
 	By default libata performs drive and controller tuning in
 	accordance with the ATA timing rules and also applies blacklists
 	and cable limits. Some controllers need special handling and have
 	custom tuning rules, typically raid controllers that use ATA
 	commands but do not actually do drive timing.
 	</para>
 	<warning>
 	<para>
 	This hook should not be used to replace the standard controller
 	tuning logic when a controller has quirks. Replacing the default
 	tuning logic in that case would bypass handling for drive and
 	bridge quirks that may be important to data reliability. If a
 	controller needs to filter the mode selection it should use the
 	mode_filter hook instead.
 	</para>
 	</warning>
 	</sect2>
 	<sect2><title>Control PCI IDE BMDMA engine</title>
 	<programlisting>
 void (*bmdma_setup) (struct ata_queued_cmd *qc);
 void (*bmdma_start) (struct ata_queued_cmd *qc);
 void (*bmdma_stop) (struct ata_port *ap);
 u8   (*bmdma_status) (struct ata_port *ap);
 	</programlisting>
 	<para>
 When setting up an IDE BMDMA transaction, these hooks arm
 (->bmdma_setup), fire (->bmdma_start), and halt (->bmdma_stop)
 the hardware's DMA engine.  ->bmdma_status is used to read the standard
 PCI IDE DMA Status register.
 	</para>
 	<para>
 These hooks are typically either no-ops, or simply not implemented, in
 FIS-based drivers.
 	</para>
 	<para>
 Most legacy IDE drivers use ata_bmdma_setup() for the bmdma_setup()
 hook.  ata_bmdma_setup() will write the pointer to the PRD table to
 the IDE PRD Table Address register, enable DMA in the DMA Command
 register, and call exec_command() to begin the transfer.
 	</para>
 	<para>
 Most legacy IDE drivers use ata_bmdma_start() for the bmdma_start()
 hook.  ata_bmdma_start() will write the ATA_DMA_START flag to the DMA
 Command register.
 	</para>
 	<para>
 Many legacy IDE drivers use ata_bmdma_stop() for the bmdma_stop()
 hook.  ata_bmdma_stop() clears the ATA_DMA_START flag in the DMA
 command register.
 	</para>
 	<para>
 Many legacy IDE drivers use ata_bmdma_status() as the bmdma_status() hook.
 	</para>
 	</sect2>
 	<sect2><title>High-level taskfile hooks</title>
 	<programlisting>
 void (*qc_prep) (struct ata_queued_cmd *qc);
 int (*qc_issue) (struct ata_queued_cmd *qc);
 	</programlisting>
 	<para>
 	Higher-level hooks, these two hooks can potentially supercede
 	several of the above taskfile/DMA engine hooks.  ->qc_prep is
 	called after the buffers have been DMA-mapped, and is typically
 	used to populate the hardware's DMA scatter-gather table.
 	Most drivers use the standard ata_qc_prep() helper function, but
 	more advanced drivers roll their own.
 	</para>
 	<para>
 	->qc_issue is used to make a command active, once the hardware
 	and S/G tables have been prepared.  IDE BMDMA drivers use the
 	helper function ata_qc_issue_prot() for taskfile protocol-based
 	dispatch.  More advanced drivers implement their own ->qc_issue.
 	</para>
 	<para>
 	ata_qc_issue_prot() calls ->tf_load(), ->bmdma_setup(), and
 	->bmdma_start() as necessary to initiate a transfer.
 	</para>
 	</sect2>
 	<sect2><title>Exception and probe handling (EH)</title>
 	<programlisting>
 void (*eng_timeout) (struct ata_port *ap);
 void (*phy_reset) (struct ata_port *ap);
 	</programlisting>
 	<para>
 Deprecated.  Use ->error_handler() instead.
 	</para>
 	<programlisting>
 void (*freeze) (struct ata_port *ap);
 void (*thaw) (struct ata_port *ap);
 	</programlisting>
 	<para>
 ata_port_freeze() is called when HSM violations or some other
 condition disrupts normal operation of the port.  A frozen port
 is not allowed to perform any operation until the port is
 thawed, which usually follows a successful reset.
 	</para>
 	<para>
 The optional ->freeze() callback can be used for freezing the port
 hardware-wise (e.g. mask interrupt and stop DMA engine).  If a
 port cannot be frozen hardware-wise, the interrupt handler
 must ack and clear interrupts unconditionally while the port
 is frozen.
 	</para>
 	<para>
 The optional ->thaw() callback is called to perform the opposite of ->freeze():
 prepare the port for normal operation once again.  Unmask interrupts,
 start DMA engine, etc.
 	</para>
 	<programlisting>
 void (*error_handler) (struct ata_port *ap);
 	</programlisting>
 	<para>
 ->error_handler() is a driver's hook into probe, hotplug, and recovery
 and other exceptional conditions.  The primary responsibility of an
 implementation is to call ata_do_eh() or ata_bmdma_drive_eh() with a set
 of EH hooks as arguments:
 	</para>
 	<para>
 'prereset' hook (may be NULL) is called during an EH reset, before any other actions
 are taken.
 	</para>
 	<para>
 'postreset' hook (may be NULL) is called after the EH reset is performed.  Based on
 existing conditions, severity of the problem, and hardware capabilities,
 	</para>
 	<para>
 Either 'softreset' (may be NULL) or 'hardreset' (may be NULL) will be
 called to perform the low-level EH reset.
 	</para>
 	<programlisting>
 void (*post_internal_cmd) (struct ata_queued_cmd *qc);
 	</programlisting>
 	<para>
 Perform any hardware-specific actions necessary to finish processing
 after executing a probe-time or EH-time command via ata_exec_internal().
 	</para>
 	</sect2>
 	<sect2><title>Hardware interrupt handling</title>
 	<programlisting>
 irqreturn_t (*irq_handler)(int, void *, struct pt_regs *);
 void (*irq_clear) (struct ata_port *);
 	</programlisting>
 	<para>
 	->irq_handler is the interrupt handling routine registered with
 	the system, by libata.  ->irq_clear is called during probe just
 	before the interrupt handler is registered, to be sure hardware
 	is quiet.
 	</para>
 	<para>
 	The second argument, dev_instance, should be cast to a pointer
 	to struct ata_host_set.
 	</para>
 	<para>
 	Most legacy IDE drivers use ata_interrupt() for the
 	irq_handler hook, which scans all ports in the host_set,
 	determines which queued command was active (if any), and calls
 	ata_host_intr(ap,qc).
 	</para>
 	<para>
 	Most legacy IDE drivers use ata_bmdma_irq_clear() for the
 	irq_clear() hook, which simply clears the interrupt and error
 	flags in the DMA status register.
 	</para>
 	</sect2>
 	<sect2><title>SATA phy read/write</title>
 	<programlisting>
 u32 (*scr_read) (struct ata_port *ap, unsigned int sc_reg);
 void (*scr_write) (struct ata_port *ap, unsigned int sc_reg,
                   u32 val);
 	</programlisting>
 	<para>
 	Read and write standard SATA phy registers.  Currently only used
 	if ->phy_reset hook called the sata_phy_reset() helper function.
 	sc_reg is one of SCR_STATUS, SCR_CONTROL, SCR_ERROR, or SCR_ACTIVE.
 	</para>
 	</sect2>
 	<sect2><title>Init and shutdown</title>
 	<programlisting>
 int (*port_start) (struct ata_port *ap);
 void (*port_stop) (struct ata_port *ap);
 void (*host_stop) (struct ata_host_set *host_set);
 	</programlisting>
 	<para>
 	->port_start() is called just after the data structures for each
 	port are initialized.  Typically this is used to alloc per-port
 	DMA buffers / tables / rings, enable DMA engines, and similar
 	tasks.  Some drivers also use this entry point as a chance to
 	allocate driver-private memory for ap->private_data.
 	</para>
 	<para>
 	Many drivers use ata_port_start() as this hook or call
 	it from their own port_start() hooks.  ata_port_start()
 	allocates space for a legacy IDE PRD table and returns.
 	</para>
 	<para>
 	->port_stop() is called after ->host_stop().  It's sole function
 	is to release DMA/memory resources, now that they are no longer
 	actively being used.  Many drivers also free driver-private
 	data from port at this time.
 	</para>
 	<para>
 	Many drivers use ata_port_stop() as this hook, which frees the
 	PRD table.
 	</para>
 	<para>
 	->host_stop() is called after all ->port_stop() calls
 have completed.  The hook must finalize hardware shutdown, release DMA
 and other resources, etc.
 	This hook may be specified as NULL, in which case it is not called.
 	</para>
 	</sect2>
     </sect1>
  </chapter>
  <chapter id="libataEH">
        <title>Error handling</title>
 	<para>
 	This chapter describes how errors are handled under libata.
 	Readers are advised to read SCSI EH
 	(Documentation/scsi/scsi_eh.txt) and ATA exceptions doc first.
 	</para>
 	<sect1><title>Origins of commands</title>
 	<para>
 	In libata, a command is represented with struct ata_queued_cmd
 	or qc.  qc's are preallocated during port initialization and
 	repetitively used for command executions.  Currently only one
 	qc is allocated per port but yet-to-be-merged NCQ branch
 	allocates one for each tag and maps each qc to NCQ tag 1-to-1.
 	</para>
 	<para>
 	libata commands can originate from two sources - libata itself
 	and SCSI midlayer.  libata internal commands are used for
 	initialization and error handling.  All normal blk requests
 	and commands for SCSI emulation are passed as SCSI commands
 	through queuecommand callback of SCSI host template.
 	</para>
 	</sect1>
 	<sect1><title>How commands are issued</title>
 	<variablelist>
 	<varlistentry><term>Internal commands</term>
 	<listitem>
 	<para>
 	First, qc is allocated and initialized using
 	ata_qc_new_init().  Although ata_qc_new_init() doesn't
 	implement any wait or retry mechanism when qc is not
 	available, internal commands are currently issued only during
 	initialization and error recovery, so no other command is
 	active and allocation is guaranteed to succeed.
 	</para>
 	<para>
 	Once allocated qc's taskfile is initialized for the command to
 	be executed.  qc currently has two mechanisms to notify
 	completion.  One is via qc->complete_fn() callback and the
 	other is completion qc->waiting.  qc->complete_fn() callback
 	is the asynchronous path used by normal SCSI translated
 	commands and qc->waiting is the synchronous (issuer sleeps in
 	process context) path used by internal commands.
 	</para>
 	<para>
 	Once initialization is complete, host_set lock is acquired
 	and the qc is issued.
 	</para>
 	</listitem>
 	</varlistentry>
 	<varlistentry><term>SCSI commands</term>
 	<listitem>
 	<para>
 	All libata drivers use ata_scsi_queuecmd() as
 	hostt->queuecommand callback.  scmds can either be simulated
 	or translated.  No qc is involved in processing a simulated
 	scmd.  The result is computed right away and the scmd is
 	completed.
 	</para>
 	<para>
 	For a translated scmd, ata_qc_new_init() is invoked to
 	allocate a qc and the scmd is translated into the qc.  SCSI
 	midlayer's completion notification function pointer is stored
 	into qc->scsidone.
 	</para>
 	<para>
 	qc->complete_fn() callback is used for completion
 	notification.  ATA commands use ata_scsi_qc_complete() while
 	ATAPI commands use atapi_qc_complete().  Both functions end up
 	calling qc->scsidone to notify upper layer when the qc is
 	finished.  After translation is completed, the qc is issued
 	with ata_qc_issue().
 	</para>
 	<para>
 	Note that SCSI midlayer invokes hostt->queuecommand while
 	holding host_set lock, so all above occur while holding
 	host_set lock.
 	</para>
 	</listitem>
 	</varlistentry>
 	</variablelist>
 	</sect1>
 	<sect1><title>How commands are processed</title>
 	<para>
 	Depending on which protocol and which controller are used,
 	commands are processed differently.  For the purpose of
 	discussion, a controller which uses taskfile interface and all
 	standard callbacks is assumed.
 	</para>
 	<para>
 	Currently 6 ATA command protocols are used.  They can be
 	sorted into the following four categories according to how
 	they are processed.
 	</para>
 	<variablelist>
 	   <varlistentry><term>ATA NO DATA or DMA</term>
 	   <listitem>
 	   <para>
 	   ATA_PROT_NODATA and ATA_PROT_DMA fall into this category.
 	   These types of commands don't require any software
 	   intervention once issued.  Device will raise interrupt on
 	   completion.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>ATA PIO</term>
 	   <listitem>
 	   <para>
 	   ATA_PROT_PIO is in this category.  libata currently
 	   implements PIO with polling.  ATA_NIEN bit is set to turn
 	   off interrupt and pio_task on ata_wq performs polling and
 	   IO.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>ATAPI NODATA or DMA</term>
 	   <listitem>
 	   <para>
 	   ATA_PROT_ATAPI_NODATA and ATA_PROT_ATAPI_DMA are in this
 	   category.  packet_task is used to poll BSY bit after
 	   issuing PACKET command.  Once BSY is turned off by the
 	   device, packet_task transfers CDB and hands off processing
 	   to interrupt handler.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>ATAPI PIO</term>
 	   <listitem>
 	   <para>
 	   ATA_PROT_ATAPI is in this category.  ATA_NIEN bit is set
 	   and, as in ATAPI NODATA or DMA, packet_task submits cdb.
 	   However, after submitting cdb, further processing (data
 	   transfer) is handed off to pio_task.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	</variablelist>
        </sect1>
 	<sect1><title>How commands are completed</title>
 	<para>
 	Once issued, all qc's are either completed with
 	ata_qc_complete() or time out.  For commands which are handled
 	by interrupts, ata_host_intr() invokes ata_qc_complete(), and,
 	for PIO tasks, pio_task invokes ata_qc_complete().  In error
 	cases, packet_task may also complete commands.
 	</para>
 	<para>
 	ata_qc_complete() does the following.
 	</para>
 	<orderedlist>
 	<listitem>
 	<para>
 	DMA memory is unmapped.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	ATA_QCFLAG_ACTIVE is clared from qc->flags.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	qc->complete_fn() callback is invoked.  If the return value of
 	the callback is not zero.  Completion is short circuited and
 	ata_qc_complete() returns.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	__ata_qc_complete() is called, which does
 	   <orderedlist>
 	   <listitem>
 	   <para>
 	   qc->flags is cleared to zero.
 	   </para>
 	   </listitem>
 	   <listitem>
 	   <para>
 	   ap->active_tag and qc->tag are poisoned.
 	   </para>
 	   </listitem>
 	   <listitem>
 	   <para>
 	   qc->waiting is claread &amp; completed (in that order).
 	   </para>
 	   </listitem>
 	   <listitem>
 	   <para>
 	   qc is deallocated by clearing appropriate bit in ap->qactive.
 	   </para>
 	   </listitem>
 	   </orderedlist>
 	</para>
 	</listitem>
 	</orderedlist>
 	<para>
 	So, it basically notifies upper layer and deallocates qc.  One
 	exception is short-circuit path in #3 which is used by
 	atapi_qc_complete().
 	</para>
 	<para>
 	For all non-ATAPI commands, whether it fails or not, almost
 	the same code path is taken and very little error handling
 	takes place.  A qc is completed with success status if it
 	succeeded, with failed status otherwise.
 	</para>
 	<para>
 	However, failed ATAPI commands require more handling as
 	REQUEST SENSE is needed to acquire sense data.  If an ATAPI
 	command fails, ata_qc_complete() is invoked with error status,
 	which in turn invokes atapi_qc_complete() via
 	qc->complete_fn() callback.
 	</para>
 	<para>
 	This makes atapi_qc_complete() set scmd->result to
 	SAM_STAT_CHECK_CONDITION, complete the scmd and return 1.  As
 	the sense data is empty but scmd->result is CHECK CONDITION,
 	SCSI midlayer will invoke EH for the scmd, and returning 1
 	makes ata_qc_complete() to return without deallocating the qc.
 	This leads us to ata_scsi_error() with partially completed qc.
 	</para>
 	</sect1>
 	<sect1><title>ata_scsi_error()</title>
 	<para>
 	ata_scsi_error() is the current transportt->eh_strategy_handler()
 	for libata.  As discussed above, this will be entered in two
 	cases - timeout and ATAPI error completion.  This function
 	calls low level libata driver's eng_timeout() callback, the
 	standard callback for which is ata_eng_timeout().  It checks
 	if a qc is active and calls ata_qc_timeout() on the qc if so.
 	Actual error handling occurs in ata_qc_timeout().
 	</para>
 	<para>
 	If EH is invoked for timeout, ata_qc_timeout() stops BMDMA and
 	completes the qc.  Note that as we're currently in EH, we
 	cannot call scsi_done.  As described in SCSI EH doc, a
 	recovered scmd should be either retried with
 	scsi_queue_insert() or finished with scsi_finish_command().
 	Here, we override qc->scsidone with scsi_finish_command() and
 	calls ata_qc_complete().
 	</para>
 	<para>
 	If EH is invoked due to a failed ATAPI qc, the qc here is
 	completed but not deallocated.  The purpose of this
 	half-completion is to use the qc as place holder to make EH
 	code reach this place.  This is a bit hackish, but it works.
 	</para>
 	<para>
 	Once control reaches here, the qc is deallocated by invoking
 	__ata_qc_complete() explicitly.  Then, internal qc for REQUEST
 	SENSE is issued.  Once sense data is acquired, scmd is
 	finished by directly invoking scsi_finish_command() on the
 	scmd.  Note that as we already have completed and deallocated
 	the qc which was associated with the scmd, we don't need
 	to/cannot call ata_qc_complete() again.
 	</para>
 	</sect1>
 	<sect1><title>Problems with the current EH</title>
 	<itemizedlist>
 	<listitem>
 	<para>
 	Error representation is too crude.  Currently any and all
 	error conditions are represented with ATA STATUS and ERROR
 	registers.  Errors which aren't ATA device errors are treated
 	as ATA device errors by setting ATA_ERR bit.  Better error
 	descriptor which can properly represent ATA and other
 	errors/exceptions is needed.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	When handling timeouts, no action is taken to make device
 	forget about the timed out command and ready for new commands.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	EH handling via ata_scsi_error() is not properly protected
 	from usual command processing.  On EH entrance, the device is
 	not in quiescent state.  Timed out commands may succeed or
 	fail any time.  pio_task and atapi_task may still be running.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Too weak error recovery.  Devices / controllers causing HSM
 	mismatch errors and other errors quite often require reset to
 	return to known state.  Also, advanced error handling is
 	necessary to support features like NCQ and hotplug.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	ATA errors are directly handled in the interrupt handler and
 	PIO errors in pio_task.  This is problematic for advanced
 	error handling for the following reasons.
 	</para>
 	<para>
 	First, advanced error handling often requires context and
 	internal qc execution.
 	</para>
 	<para>
 	Second, even a simple failure (say, CRC error) needs
 	information gathering and could trigger complex error handling
 	(say, resetting &amp; reconfiguring).  Having multiple code
 	paths to gather information, enter EH and trigger actions
 	makes life painful.
 	</para>
 	<para>
 	Third, scattered EH code makes implementing low level drivers
 	difficult.  Low level drivers override libata callbacks.  If
 	EH is scattered over several places, each affected callbacks
 	should perform its part of error handling.  This can be error
 	prone and painful.
 	</para>
 	</listitem>
 	</itemizedlist>
 	</sect1>
  </chapter>
  <chapter id="libataExt">
     <title>libata Library</title>
 !Edrivers/ata/libata-core.c
  </chapter>
  <chapter id="libataInt">
     <title>libata Core Internals</title>
 !Idrivers/ata/libata-core.c
  </chapter>
  <chapter id="libataScsiInt">
     <title>libata SCSI translation/emulation</title>
 !Edrivers/ata/libata-scsi.c
 !Idrivers/ata/libata-scsi.c
  </chapter>
  <chapter id="ataExceptions">
     <title>ATA errors &amp; exceptions</title>
  <para>
  This chapter tries to identify what error/exception conditions exist
  for ATA/ATAPI devices and describe how they should be handled in
  implementation-neutral way.
  </para>
  <para>
  The term 'error' is used to describe conditions where either an
  explicit error condition is reported from device or a command has
  timed out.
  </para>
  <para>
  The term 'exception' is either used to describe exceptional
  conditions which are not errors (say, power or hotplug events), or
  to describe both errors and non-error exceptional conditions.  Where
  explicit distinction between error and exception is necessary, the
  term 'non-error exception' is used.
  </para>
  <sect1 id="excat">
     <title>Exception categories</title>
     <para>
     Exceptions are described primarily with respect to legacy
     taskfile + bus master IDE interface.  If a controller provides
     other better mechanism for error reporting, mapping those into
     categories described below shouldn't be difficult.
     </para>
     <para>
     In the following sections, two recovery actions - reset and
     reconfiguring transport - are mentioned.  These are described
     further in <xref linkend="exrec"/>.
     </para>
     <sect2 id="excatHSMviolation">
        <title>HSM violation</title>
        <para>
        This error is indicated when STATUS value doesn't match HSM
        requirement during issuing or excution any ATA/ATAPI command.
        </para>
 	<itemizedlist>
 	<title>Examples</title>
        <listitem>
 	<para>
 	ATA_STATUS doesn't contain !BSY &amp;&amp; DRDY &amp;&amp; !DRQ while trying
 	to issue a command.
        </para>
 	</listitem>
        <listitem>
 	<para>
 	!BSY &amp;&amp; !DRQ during PIO data transfer.
        </para>
 	</listitem>
        <listitem>
 	<para>
 	DRQ on command completion.
        </para>
 	</listitem>
        <listitem>
 	<para>
 	!BSY &amp;&amp; ERR after CDB tranfer starts but before the
        last byte of CDB is transferred.  ATA/ATAPI standard states
        that &quot;The device shall not terminate the PACKET command
        with an error before the last byte of the command packet has
        been written&quot; in the error outputs description of PACKET
        command and the state diagram doesn't include such
        transitions.
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	In these cases, HSM is violated and not much information
 	regarding the error can be acquired from STATUS or ERROR
 	register.  IOW, this error can be anything - driver bug,
 	faulty device, controller and/or cable.
 	</para>
 	<para>
 	As HSM is violated, reset is necessary to restore known state.
 	Reconfiguring transport for lower speed might be helpful too
 	as transmission errors sometimes cause this kind of errors.
 	</para>
     </sect2>
     <sect2 id="excatDevErr">
        <title>ATA/ATAPI device error (non-NCQ / non-CHECK CONDITION)</title>
 	<para>
 	These are errors detected and reported by ATA/ATAPI devices
 	indicating device problems.  For this type of errors, STATUS
 	and ERROR register values are valid and describe error
 	condition.  Note that some of ATA bus errors are detected by
 	ATA/ATAPI devices and reported using the same mechanism as
 	device errors.  Those cases are described later in this
 	section.
 	</para>
 	<para>
 	For ATA commands, this type of errors are indicated by !BSY
 	&amp;&amp; ERR during command execution and on completion.
 	</para>
 	<para>For ATAPI commands,</para>
 	<itemizedlist>
 	<listitem>
 	<para>
 	!BSY &amp;&amp; ERR &amp;&amp; ABRT right after issuing PACKET
 	indicates that PACKET command is not supported and falls in
 	this category.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	!BSY &amp;&amp; ERR(==CHK) &amp;&amp; !ABRT after the last
 	byte of CDB is transferred indicates CHECK CONDITION and
 	doesn't fall in this category.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	!BSY &amp;&amp; ERR(==CHK) &amp;&amp; ABRT after the last byte
        of CDB is transferred *probably* indicates CHECK CONDITION and
        doesn't fall in this category.
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	Of errors detected as above, the followings are not ATA/ATAPI
 	device errors but ATA bus errors and should be handled
 	according to <xref linkend="excatATAbusErr"/>.
 	</para>
 	<variablelist>
 	   <varlistentry>
 	   <term>CRC error during data transfer</term>
 	   <listitem>
 	   <para>
 	   This is indicated by ICRC bit in the ERROR register and
 	   means that corruption occurred during data transfer.  Upto
 	   ATA/ATAPI-7, the standard specifies that this bit is only
 	   applicable to UDMA transfers but ATA/ATAPI-8 draft revision
 	   1f says that the bit may be applicable to multiword DMA and
 	   PIO.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry>
 	   <term>ABRT error during data transfer or on completion</term>
 	   <listitem>
 	   <para>
 	   Upto ATA/ATAPI-7, the standard specifies that ABRT could be
 	   set on ICRC errors and on cases where a device is not able
 	   to complete a command.  Combined with the fact that MWDMA
 	   and PIO transfer errors aren't allowed to use ICRC bit upto
 	   ATA/ATAPI-7, it seems to imply that ABRT bit alone could
 	   indicate tranfer errors.
 	   </para>
 	   <para>
 	   However, ATA/ATAPI-8 draft revision 1f removes the part
 	   that ICRC errors can turn on ABRT.  So, this is kind of
 	   gray area.  Some heuristics are needed here.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	</variablelist>
 	<para>
 	ATA/ATAPI device errors can be further categorized as follows.
 	</para>
 	<variablelist>
 	   <varlistentry>
 	   <term>Media errors</term>
 	   <listitem>
 	   <para>
 	   This is indicated by UNC bit in the ERROR register.  ATA
 	   devices reports UNC error only after certain number of
 	   retries cannot recover the data, so there's nothing much
 	   else to do other than notifying upper layer.
 	   </para>
 	   <para>
 	   READ and WRITE commands report CHS or LBA of the first
 	   failed sector but ATA/ATAPI standard specifies that the
 	   amount of transferred data on error completion is
 	   indeterminate, so we cannot assume that sectors preceding
 	   the failed sector have been transferred and thus cannot
 	   complete those sectors successfully as SCSI does.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry>
 	   <term>Media changed / media change requested error</term>
 	   <listitem>
 	   <para>
 	   &lt;&lt;TODO: fill here&gt;&gt;
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>Address error</term>
 	   <listitem>
 	   <para>
 	   This is indicated by IDNF bit in the ERROR register.
 	   Report to upper layer.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>Other errors</term>
 	   <listitem>
 	   <para>
 	   This can be invalid command or parameter indicated by ABRT
 	   ERROR bit or some other error condition.  Note that ABRT
 	   bit can indicate a lot of things including ICRC and Address
 	   errors.  Heuristics needed.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	</variablelist>
 	<para>
 	Depending on commands, not all STATUS/ERROR bits are
 	applicable.  These non-applicable bits are marked with
 	&quot;na&quot; in the output descriptions but upto ATA/ATAPI-7
 	no definition of &quot;na&quot; can be found.  However,
 	ATA/ATAPI-8 draft revision 1f describes &quot;N/A&quot; as
 	follows.
 	</para>
 	<blockquote>
 	<variablelist>
 	   <varlistentry><term>3.2.3.3a N/A</term>
 	   <listitem>
 	   <para>
 	   A keyword the indicates a field has no defined value in
 	   this standard and should not be checked by the host or
 	   device. N/A fields should be cleared to zero.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	</variablelist>
 	</blockquote>
 	<para>
 	So, it seems reasonable to assume that &quot;na&quot; bits are
 	cleared to zero by devices and thus need no explicit masking.
 	</para>
     </sect2>
     <sect2 id="excatATAPIcc">
        <title>ATAPI device CHECK CONDITION</title>
 	<para>
 	ATAPI device CHECK CONDITION error is indicated by set CHK bit
 	(ERR bit) in the STATUS register after the last byte of CDB is
 	transferred for a PACKET command.  For this kind of errors,
 	sense data should be acquired to gather information regarding
 	the errors.  REQUEST SENSE packet command should be used to
 	acquire sense data.
 	</para>
 	<para>
 	Once sense data is acquired, this type of errors can be
 	handled similary to other SCSI errors.  Note that sense data
 	may indicate ATA bus error (e.g. Sense Key 04h HARDWARE ERROR
 	&amp;&amp; ASC/ASCQ 47h/00h SCSI PARITY ERROR).  In such
 	cases, the error should be considered as an ATA bus error and
 	handled according to <xref linkend="excatATAbusErr"/>.
 	</para>
     </sect2>
     <sect2 id="excatNCQerr">
        <title>ATA device error (NCQ)</title>
 	<para>
 	NCQ command error is indicated by cleared BSY and set ERR bit
 	during NCQ command phase (one or more NCQ commands
 	outstanding).  Although STATUS and ERROR registers will
 	contain valid values describing the error, READ LOG EXT is
 	required to clear the error condition, determine which command
 	has failed and acquire more information.
 	</para>
 	<para>
 	READ LOG EXT Log Page 10h reports which tag has failed and
 	taskfile register values describing the error.  With this
 	information the failed command can be handled as a normal ATA
 	command error as in <xref linkend="excatDevErr"/> and all
 	other in-flight commands must be retried.  Note that this
 	retry should not be counted - it's likely that commands
 	retried this way would have completed normally if it were not
 	for the failed command.
 	</para>
 	<para>
 	Note that ATA bus errors can be reported as ATA device NCQ
 	errors.  This should be handled as described in <xref
 	linkend="excatATAbusErr"/>.
 	</para>
 	<para>
 	If READ LOG EXT Log Page 10h fails or reports NQ, we're
 	thoroughly screwed.  This condition should be treated
 	according to <xref linkend="excatHSMviolation"/>.
 	</para>
     </sect2>
     <sect2 id="excatATAbusErr">
        <title>ATA bus error</title>
 	<para>
 	ATA bus error means that data corruption occurred during
 	transmission over ATA bus (SATA or PATA).  This type of errors
 	can be indicated by
 	</para>
 	<itemizedlist>
 	<listitem>
 	<para>
 	ICRC or ABRT error as described in <xref linkend="excatDevErr"/>.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Controller-specific error completion with error information
 	indicating transmission error.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	On some controllers, command timeout.  In this case, there may
 	be a mechanism to determine that the timeout is due to
 	transmission error.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Unknown/random errors, timeouts and all sorts of weirdities.
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	As described above, transmission errors can cause wide variety
 	of symptoms ranging from device ICRC error to random device
 	lockup, and, for many cases, there is no way to tell if an
 	error condition is due to transmission error or not;
 	therefore, it's necessary to employ some kind of heuristic
 	when dealing with errors and timeouts.  For example,
 	encountering repetitive ABRT errors for known supported
 	command is likely to indicate ATA bus error.
 	</para>
 	<para>
 	Once it's determined that ATA bus errors have possibly
 	occurred, lowering ATA bus transmission speed is one of
 	actions which may alleviate the problem.  See <xref
 	linkend="exrecReconf"/> for more information.
 	</para>
     </sect2>
     <sect2 id="excatPCIbusErr">
        <title>PCI bus error</title>
 	<para>
 	Data corruption or other failures during transmission over PCI
 	(or other system bus).  For standard BMDMA, this is indicated
 	by Error bit in the BMDMA Status register.  This type of
 	errors must be logged as it indicates something is very wrong
 	with the system.  Resetting host controller is recommended.
 	</para>
     </sect2>
     <sect2 id="excatLateCompletion">
        <title>Late completion</title>
 	<para>
 	This occurs when timeout occurs and the timeout handler finds
 	out that the timed out command has completed successfully or
 	with error.  This is usually caused by lost interrupts.  This
 	type of errors must be logged.  Resetting host controller is
 	recommended.
 	</para>
     </sect2>
     <sect2 id="excatUnknown">
        <title>Unknown error (timeout)</title>
 	<para>
 	This is when timeout occurs and the command is still
 	processing or the host and device are in unknown state.  When
 	this occurs, HSM could be in any valid or invalid state.  To
 	bring the device to known state and make it forget about the
 	timed out command, resetting is necessary.  The timed out
 	command may be retried.
 	</para>
 	<para>
 	Timeouts can also be caused by transmission errors.  Refer to
 	<xref linkend="excatATAbusErr"/> for more details.
 	</para>
     </sect2>
     <sect2 id="excatHoplugPM">
        <title>Hotplug and power management exceptions</title>
 	<para>
 	&lt;&lt;TODO: fill here&gt;&gt;
 	</para>
     </sect2>
  </sect1>
  <sect1 id="exrec">
     <title>EH recovery actions</title>
     <para>
     This section discusses several important recovery actions.
     </para>
     <sect2 id="exrecClr">
        <title>Clearing error condition</title>
 	<para>
 	Many controllers require its error registers to be cleared by
 	error handler.  Different controllers may have different
 	requirements.
 	</para>
 	<para>
 	For SATA, it's strongly recommended to clear at least SError
 	register during error handling.
 	</para>
     </sect2>
     <sect2 id="exrecRst">
        <title>Reset</title>
 	<para>
 	During EH, resetting is necessary in the following cases.
 	</para>
 	<itemizedlist>
 	<listitem>
 	<para>
 	HSM is in unknown or invalid state
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	HBA is in unknown or invalid state
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	EH needs to make HBA/device forget about in-flight commands
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	HBA/device behaves weirdly
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	Resetting during EH might be a good idea regardless of error
 	condition to improve EH robustness.  Whether to reset both or
 	either one of HBA and device depends on situation but the
 	following scheme is recommended.
 	</para>
 	<itemizedlist>
 	<listitem>
 	<para>
 	When it's known that HBA is in ready state but ATA/ATAPI
-	device in in unknown state, reset only device.
+	device is in unknown state, reset only device.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	If HBA is in unknown state, reset both HBA and device.
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	HBA resetting is implementation specific.  For a controller
 	complying to taskfile/BMDMA PCI IDE, stopping active DMA
 	transaction may be sufficient iff BMDMA state is the only HBA
 	context.  But even mostly taskfile/BMDMA PCI IDE complying
 	controllers may have implementation specific requirements and
 	mechanism to reset themselves.  This must be addressed by
 	specific drivers.
 	</para>
 	<para>
 	OTOH, ATA/ATAPI standard describes in detail ways to reset
 	ATA/ATAPI devices.
 	</para>
 	<variablelist>
 	   <varlistentry><term>PATA hardware reset</term>
 	   <listitem>
 	   <para>
 	   This is hardware initiated device reset signalled with
 	   asserted PATA RESET- signal.  There is no standard way to
 	   initiate hardware reset from software although some
 	   hardware provides registers that allow driver to directly
 	   tweak the RESET- signal.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>Software reset</term>
 	   <listitem>
 	   <para>
 	   This is achieved by turning CONTROL SRST bit on for at
 	   least 5us.  Both PATA and SATA support it but, in case of
 	   SATA, this may require controller-specific support as the
 	   second Register FIS to clear SRST should be transmitted
 	   while BSY bit is still set.  Note that on PATA, this resets
 	   both master and slave devices on a channel.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>EXECUTE DEVICE DIAGNOSTIC command</term>
 	   <listitem>
 	   <para>
 	   Although ATA/ATAPI standard doesn't describe exactly, EDD
 	   implies some level of resetting, possibly similar level
 	   with software reset.  Host-side EDD protocol can be handled
 	   with normal command processing and most SATA controllers
 	   should be able to handle EDD's just like other commands.
 	   As in software reset, EDD affects both devices on a PATA
 	   bus.
 	   </para>
 	   <para>
 	   Although EDD does reset devices, this doesn't suit error
 	   handling as EDD cannot be issued while BSY is set and it's
 	   unclear how it will act when device is in unknown/weird
 	   state.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>ATAPI DEVICE RESET command</term>
 	   <listitem>
 	   <para>
 	   This is very similar to software reset except that reset
 	   can be restricted to the selected device without affecting
 	   the other device sharing the cable.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	   <varlistentry><term>SATA phy reset</term>
 	   <listitem>
 	   <para>
 	   This is the preferred way of resetting a SATA device.  In
 	   effect, it's identical to PATA hardware reset.  Note that
 	   this can be done with the standard SCR Control register.
 	   As such, it's usually easier to implement than software
 	   reset.
 	   </para>
 	   </listitem>
 	   </varlistentry>
 	</variablelist>
 	<para>
 	One more thing to consider when resetting devices is that
 	resetting clears certain configuration parameters and they
 	need to be set to their previous or newly adjusted values
 	after reset.
 	</para>
 	<para>
 	Parameters affected are.
 	</para>
 	<itemizedlist>
 	<listitem>
 	<para>
 	CHS set up with INITIALIZE DEVICE PARAMETERS (seldomly used)
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Parameters set with SET FEATURES including transfer mode setting
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Block count set with SET MULTIPLE MODE
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Other parameters (SET MAX, MEDIA LOCK...)
 	</para>
 	</listitem>
 	</itemizedlist>
 	<para>
 	ATA/ATAPI standard specifies that some parameters must be
 	maintained across hardware or software reset, but doesn't
 	strictly specify all of them.  Always reconfiguring needed
 	parameters after reset is required for robustness.  Note that
 	this also applies when resuming from deep sleep (power-off).
 	</para>
 	<para>
 	Also, ATA/ATAPI standard requires that IDENTIFY DEVICE /
 	IDENTIFY PACKET DEVICE is issued after any configuration
 	parameter is updated or a hardware reset and the result used
 	for further operation.  OS driver is required to implement
 	revalidation mechanism to support this.
 	</para>
     </sect2>
     <sect2 id="exrecReconf">
        <title>Reconfigure transport</title>
 	<para>
 	For both PATA and SATA, a lot of corners are cut for cheap
 	connectors, cables or controllers and it's quite common to see
 	high transmission error rate.  This can be mitigated by
 	lowering transmission speed.
 	</para>
 	<para>
 	The following is a possible scheme Jeff Garzik suggested.
 	</para>
 	<blockquote>
 	<para>
 	If more than $N (3?) transmission errors happen in 15 minutes,
 	</para>	
 	<itemizedlist>
 	<listitem>
 	<para>
 	if SATA, decrease SATA PHY speed.  if speed cannot be decreased,
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	decrease UDMA xfer speed.  if at UDMA0, switch to PIO4,
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	decrease PIO xfer speed.  if at PIO3, complain, but continue
 	</para>
 	</listitem>
 	</itemizedlist>
 	</blockquote>
     </sect2>
  </sect1>
  </chapter>
  <chapter id="PiixInt">
     <title>ata_piix Internals</title>
 !Idrivers/ata/ata_piix.c
  </chapter>
  <chapter id="SILInt">
     <title>sata_sil Internals</title>
 !Idrivers/ata/sata_sil.c
  </chapter>
  <chapter id="libataThanks">
     <title>Thanks</title>
  <para>
  The bulk of the ATA knowledge comes thanks to long conversations with
  Andre Hedrick (www.linux-ide.org), and long hours pondering the ATA
  and SCSI specifications.
  </para>
  <para>
  Thanks to Alan Cox for pointing out similarities 
  between SATA and SCSI, and in general for motivation to hack on
  libata.
  </para>
  <para>
  libata's device detection
  method, ata_pio_devchk, and in general all the early probing was
  based on extensive study of Hale Landis's probe/reset code in his
  ATADRVR driver (www.ata-atapi.com).
  </para>
  </chapter>
 </book>
 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
 	"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
 <book id="Linux-USB-API">
 <bookinfo>
  <title>The Linux-USB Host Side API</title>
  <legalnotice>
   <para>
     This documentation is free software; you can redistribute
     it and/or modify it under the terms of the GNU General Public
     License as published by the Free Software Foundation; either
     version 2 of the License, or (at your option) any later
     version.
   </para>
   <para>
     This program is distributed in the hope that it will be
     useful, but WITHOUT ANY WARRANTY; without even the implied
     warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
     See the GNU General Public License for more details.
   </para>
   <para>
     You should have received a copy of the GNU General Public
     License along with this program; if not, write to the Free
     Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
     MA 02111-1307 USA
   </para>
   <para>
     For more details see the file COPYING in the source
     distribution of Linux.
   </para>
  </legalnotice>
 </bookinfo>
 <toc></toc>
 <chapter id="intro">
    <title>Introduction to USB on Linux</title>
    <para>A Universal Serial Bus (USB) is used to connect a host,
    such as a PC or workstation, to a number of peripheral
    devices.  USB uses a tree structure, with the host as the
    root (the system's master), hubs as interior nodes, and
    peripherals as leaves (and slaves).
    Modern PCs support several such trees of USB devices, usually
    one USB 2.0 tree (480 Mbit/sec each) with
    a few USB 1.1 trees (12 Mbit/sec each) that are used when you
    connect a USB 1.1 device directly to the machine's "root hub".
    </para>
    <para>That master/slave asymmetry was designed-in for a number of
    reasons, one being ease of use.  It is not physically possible to
    assemble (legal) USB cables incorrectly:  all upstream "to the host"
    connectors are the rectangular type (matching the sockets on
    root hubs), and all downstream connectors are the squarish type
    (or they are built into the peripheral).
    Also, the host software doesn't need to deal with distributed
    auto-configuration since the pre-designated master node manages all that.
    And finally, at the electrical level, bus protocol overhead is reduced by
    eliminating arbitration and moving scheduling into the host software.
    </para>
    <para>USB 1.0 was announced in January 1996 and was revised
    as USB 1.1 (with improvements in hub specification and
    support for interrupt-out transfers) in September 1998.
    USB 2.0 was released in April 2000, adding high-speed
    transfers and transaction-translating hubs (used for USB 1.1
    and 1.0 backward compatibility).
    </para>
    <para>Kernel developers added USB support to Linux early in the 2.2 kernel
    series, shortly before 2.3 development forked.  Updates from 2.3 were
    regularly folded back into 2.2 releases, which improved reliability and
    brought <filename>/sbin/hotplug</filename> support as well more drivers.
    Such improvements were continued in the 2.5 kernel series, where they added
    USB 2.0 support, improved performance, and made the host controller drivers
    (HCDs) more consistent.  They also simplified the API (to make bugs less
    likely) and added internal "kerneldoc" documentation.
    </para>
    <para>Linux can run inside USB devices as well as on
    the hosts that control the devices.
    But USB device drivers running inside those peripherals
    don't do the same things as the ones running inside hosts,
    so they've been given a different name:
    <emphasis>gadget drivers</emphasis>.
    This document does not cover gadget drivers.
    </para>
    </chapter>
 <chapter id="host">
    <title>USB Host-Side API Model</title>
    <para>Host-side drivers for USB devices talk to the "usbcore" APIs.
    There are two.  One is intended for
    <emphasis>general-purpose</emphasis> drivers (exposed through
    driver frameworks), and the other is for drivers that are
    <emphasis>part of the core</emphasis>.
    Such core drivers include the <emphasis>hub</emphasis> driver
    (which manages trees of USB devices) and several different kinds
    of <emphasis>host controller drivers</emphasis>,
    which control individual busses.
    </para>
    <para>The device model seen by USB drivers is relatively complex.
    </para>
    <itemizedlist>
 	<listitem><para>USB supports four kinds of data transfers
 	(control, bulk, interrupt, and isochronous).  Two of them (control
 	and bulk) use bandwidth as it's available,
 	while the other two (interrupt and isochronous)
 	are scheduled to provide guaranteed bandwidth.
 	</para></listitem>
 	<listitem><para>The device description model includes one or more
 	"configurations" per device, only one of which is active at a time.
 	Devices that are capable of high-speed operation must also support
 	full-speed configurations, along with a way to ask about the
 	"other speed" configurations which might be used.
 	</para></listitem>
 	<listitem><para>Configurations have one or more "interfaces", each
 	of which may have "alternate settings".  Interfaces may be
 	standardized by USB "Class" specifications, or may be specific to
 	a vendor or device.</para>
 	<para>USB device drivers actually bind to interfaces, not devices.
 	Think of them as "interface drivers", though you
 	may not see many devices where the distinction is important.
 	<emphasis>Most USB devices are simple, with only one configuration,
 	one interface, and one alternate setting.</emphasis>
 	</para></listitem>
 	<listitem><para>Interfaces have one or more "endpoints", each of
 	which supports one type and direction of data transfer such as
 	"bulk out" or "interrupt in".  The entire configuration may have
 	up to sixteen endpoints in each direction, allocated as needed
 	among all the interfaces.
 	</para></listitem>
 	<listitem><para>Data transfer on USB is packetized; each endpoint
 	has a maximum packet size.
 	Drivers must often be aware of conventions such as flagging the end
 	of bulk transfers using "short" (including zero length) packets.
 	</para></listitem>
 	<listitem><para>The Linux USB API supports synchronous calls for
 	control and bulk messages.
 	It also supports asynchnous calls for all kinds of data transfer,
 	using request structures called "URBs" (USB Request Blocks).
 	</para></listitem>
    </itemizedlist>
    <para>Accordingly, the USB Core API exposed to device drivers
    covers quite a lot of territory.  You'll probably need to consult
    the USB 2.0 specification, available online from www.usb.org at
    no cost, as well as class or device specifications.
    </para>
    <para>The only host-side drivers that actually touch hardware
    (reading/writing registers, handling IRQs, and so on) are the HCDs.
    In theory, all HCDs provide the same functionality through the same
    API.  In practice, that's becoming more true on the 2.5 kernels,
    but there are still differences that crop up especially with
    fault handling.  Different controllers don't necessarily report
    the same aspects of failures, and recovery from faults (including
    software-induced ones like unlinking an URB) isn't yet fully
    consistent.
    Device driver authors should make a point of doing disconnect
    testing (while the device is active) with each different host
    controller driver, to make sure drivers don't have bugs of
    their own as well as to make sure they aren't relying on some
    HCD-specific behavior.
    (You will need external USB 1.1 and/or
    USB 2.0 hubs to perform all those tests.)
    </para>
    </chapter>
 <chapter><title>USB-Standard Types</title>
    <para>In <filename>&lt;linux/usb_ch9.h&gt;</filename> you will find
    the USB data types defined in chapter 9 of the USB specification.
    These data types are used throughout USB, and in APIs including
    this host side API, gadget APIs, and usbfs.
    </para>
 !Iinclude/linux/usb_ch9.h
    </chapter>
 <chapter><title>Host-Side Data Types and Macros</title>
    <para>The host side API exposes several layers to drivers, some of
    which are more necessary than others.
    These support lifecycle models for host side drivers
    and devices, and support passing buffers through usbcore to
    some HCD that performs the I/O for the device driver.
    </para>
 !Iinclude/linux/usb.h
    </chapter>
    <chapter><title>USB Core APIs</title>
    <para>There are two basic I/O models in the USB API.
    The most elemental one is asynchronous:  drivers submit requests
    in the form of an URB, and the URB's completion callback
    handle the next step.
    All USB transfer types support that model, although there
    are special cases for control URBs (which always have setup
    and status stages, but may not have a data stage) and
    isochronous URBs (which allow large packets and include
    per-packet fault reports).
    Built on top of that is synchronous API support, where a
    driver calls a routine that allocates one or more URBs,
    submits them, and waits until they complete.
    There are synchronous wrappers for single-buffer control
    and bulk transfers (which are awkward to use in some
    driver disconnect scenarios), and for scatterlist based
    streaming i/o (bulk or interrupt).
    </para>
    <para>USB drivers need to provide buffers that can be
    used for DMA, although they don't necessarily need to
    provide the DMA mapping themselves.
    There are APIs to use used when allocating DMA buffers,
    which can prevent use of bounce buffers on some systems.
    In some cases, drivers may be able to rely on 64bit DMA
    to eliminate another kind of bounce buffer.
    </para>
 !Edrivers/usb/core/urb.c
 !Edrivers/usb/core/message.c
 !Edrivers/usb/core/file.c
 !Edrivers/usb/core/driver.c
 !Edrivers/usb/core/usb.c
 !Edrivers/usb/core/hub.c
    </chapter>
    <chapter><title>Host Controller APIs</title>
    <para>These APIs are only for use by host controller drivers,
    most of which implement standard register interfaces such as
    EHCI, OHCI, or UHCI.
    UHCI was one of the first interfaces, designed by Intel and
    also used by VIA; it doesn't do much in hardware.
    OHCI was designed later, to have the hardware do more work
    (bigger transfers, tracking protocol state, and so on).
    EHCI was designed with USB 2.0; its design has features that
    resemble OHCI (hardware does much more work) as well as
    UHCI (some parts of ISO support, TD list processing).
    </para>
    <para>There are host controllers other than the "big three",
    although most PCI based controllers (and a few non-PCI based
    ones) use one of those interfaces.
    Not all host controllers use DMA; some use PIO, and there
    is also a simulator.
    </para>
    <para>The same basic APIs are available to drivers for all
    those controllers.  
    For historical reasons they are in two layers:
    <structname>struct usb_bus</structname> is a rather thin
    layer that became available in the 2.2 kernels, while
    <structname>struct usb_hcd</structname> is a more featureful
    layer (available in later 2.4 kernels and in 2.5) that
    lets HCDs share common code, to shrink driver size
    and significantly reduce hcd-specific behaviors.
    </para>
 !Edrivers/usb/core/hcd.c
 !Edrivers/usb/core/hcd-pci.c
 !Idrivers/usb/core/buffer.c
    </chapter>
    <chapter>
 	<title>The USB Filesystem (usbfs)</title>
 	<para>This chapter presents the Linux <emphasis>usbfs</emphasis>.
 	You may prefer to avoid writing new kernel code for your
 	USB driver; that's the problem that usbfs set out to solve.
 	User mode device drivers are usually packaged as applications
 	or libraries, and may use usbfs through some programming library
 	that wraps it.  Such libraries include
 	<ulink url="http://libusb.sourceforge.net">libusb</ulink>
 	for C/C++, and
 	<ulink url="http://jUSB.sourceforge.net">jUSB</ulink> for Java.
 	</para>
 	<note><title>Unfinished</title>
 	    <para>This particular documentation is incomplete,
 	    especially with respect to the asynchronous mode.
 	    As of kernel 2.5.66 the code and this (new) documentation
 	    need to be cross-reviewed.
 	    </para>
 	    </note>
 	<para>Configure usbfs into Linux kernels by enabling the
 	<emphasis>USB filesystem</emphasis> option (CONFIG_USB_DEVICEFS),
 	and you get basic support for user mode USB device drivers.
 	Until relatively recently it was often (confusingly) called
 	<emphasis>usbdevfs</emphasis> although it wasn't solving what
 	<emphasis>devfs</emphasis> was.
 	Every USB device will appear in usbfs, regardless of whether or
 	not it has a kernel driver.
 	</para>
 	<sect1>
 	    <title>What files are in "usbfs"?</title>
 	    <para>Conventionally mounted at
 	    <filename>/proc/bus/usb</filename>, usbfs 
 	    features include:
 	    <itemizedlist>
 		<listitem><para><filename>/proc/bus/usb/devices</filename>
 		    ... a text file
 		    showing each of the USB devices on known to the kernel,
 		    and their configuration descriptors.
 		    You can also poll() this to learn about new devices.
 		    </para></listitem>
 		<listitem><para><filename>/proc/bus/usb/BBB/DDD</filename>
 		    ... magic files
 		    exposing the each device's configuration descriptors, and
 		    supporting a series of ioctls for making device requests,
 		    including I/O to devices.  (Purely for access by programs.)
 		    </para></listitem>
 	    </itemizedlist>
 	    </para>
 	    <para> Each bus is given a number (BBB) based on when it was
 	    enumerated; within each bus, each device is given a similar
 	    number (DDD).
 	    Those BBB/DDD paths are not "stable" identifiers;
 	    expect them to change even if you always leave the devices
 	    plugged in to the same hub port.
 	    <emphasis>Don't even think of saving these in application
 	    configuration files.</emphasis>
 	    Stable identifiers are available, for user mode applications
 	    that want to use them.  HID and networking devices expose
 	    these stable IDs, so that for example you can be sure that
 	    you told the right UPS to power down its second server.
 	    "usbfs" doesn't (yet) expose those IDs.
 	    </para>
 	</sect1>
 	<sect1>
 	    <title>Mounting and Access Control</title>
 	    <para>There are a number of mount options for usbfs, which will
 	    be of most interest to you if you need to override the default
 	    access control policy.
 	    That policy is that only root may read or write device files
 	    (<filename>/proc/bus/BBB/DDD</filename>) although anyone may read
 	    the <filename>devices</filename>
 	    or <filename>drivers</filename> files.
 	    I/O requests to the device also need the CAP_SYS_RAWIO capability,
 	    </para>
 	    <para>The significance of that is that by default, all user mode
 	    device drivers need super-user privileges.
 	    You can change modes or ownership in a driver setup
 	    when the device hotplugs, or maye just start the
 	    driver right then, as a privileged server (or some activity
 	    within one).
 	    That's the most secure approach for multi-user systems,
 	    but for single user systems ("trusted" by that user)
 	    it's more convenient just to grant everyone all access
 	    (using the <emphasis>devmode=0666</emphasis> option)
 	    so the driver can start whenever it's needed.
 	    </para>
 	    <para>The mount options for usbfs, usable in /etc/fstab or
 	    in command line invocations of <emphasis>mount</emphasis>, are:
 	    <variablelist>
 		<varlistentry>
 		    <term><emphasis>busgid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the GID used for the
 		    /proc/bus/usb/BBB
 		    directories.  (Default: 0)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>busmode</emphasis>=MMM</term>
 		    <listitem><para>Controls the file mode used for the
 		    /proc/bus/usb/BBB
 		    directories.  (Default: 0555)
 		    </para></listitem></varlistentry>
 		<varlistentry><term><emphasis>busuid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the UID used for the
 		    /proc/bus/usb/BBB
 		    directories.  (Default: 0)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>devgid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the GID used for the
 		    /proc/bus/usb/BBB/DDD
 		    files.  (Default: 0)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>devmode</emphasis>=MMM</term>
 		    <listitem><para>Controls the file mode used for the
 		    /proc/bus/usb/BBB/DDD
 		    files.  (Default: 0644)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>devuid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the UID used for the
 		    /proc/bus/usb/BBB/DDD
 		    files.  (Default: 0)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>listgid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the GID used for the
 		    /proc/bus/usb/devices and drivers files.
 		    (Default: 0)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>listmode</emphasis>=MMM</term>
 		    <listitem><para>Controls the file mode used for the
 		    /proc/bus/usb/devices and drivers files.
 		    (Default: 0444)</para></listitem></varlistentry>
 		<varlistentry><term><emphasis>listuid</emphasis>=NNNNN</term>
 		    <listitem><para>Controls the UID used for the
 		    /proc/bus/usb/devices and drivers files.
 		    (Default: 0)</para></listitem></varlistentry>
 	    </variablelist>
 	    </para>
 	    <para>Note that many Linux distributions hard-wire the mount options
 	    for usbfs in their init scripts, such as
 	    <filename>/etc/rc.d/rc.sysinit</filename>,
 	    rather than making it easy to set this per-system
 	    policy in <filename>/etc/fstab</filename>.
 	    </para>
 	</sect1>
 	<sect1>
 	    <title>/proc/bus/usb/devices</title>
 	    <para>This file is handy for status viewing tools in user
 	    mode, which can scan the text format and ignore most of it.
 	    More detailed device status (including class and vendor
 	    status) is available from device-specific files.
 	    For information about the current format of this file,
 	    see the
 	    <filename>Documentation/usb/proc_usb_info.txt</filename>
 	    file in your Linux kernel sources.
 	    </para>
 	    <para>This file, in combination with the poll() system call, can
 	    also be used to detect when devices are added or removed:
 <programlisting>int fd;
 struct pollfd pfd;
 fd = open("/proc/bus/usb/devices", O_RDONLY);
 pfd = { fd, POLLIN, 0 };
 for (;;) {
 	/* The first time through, this call will return immediately. */
 	poll(&amp;pfd, 1, -1);
 	/* To see what's changed, compare the file's previous and current
 	   contents or scan the filesystem.  (Scanning is more precise.) */
 }</programlisting>
 	    Note that this behavior is intended to be used for informational
 	    and debug purposes.  It would be more appropriate to use programs
 	    such as udev or HAL to initialize a device or start a user-mode
 	    helper program, for instance.
 	    </para>
 	</sect1>
 	<sect1>
 	    <title>/proc/bus/usb/BBB/DDD</title>
 	    <para>Use these files in one of these basic ways:
 	    </para>
 	    <para><emphasis>They can be read,</emphasis>
 	    producing first the device descriptor
 	    (18 bytes) and then the descriptors for the current configuration.
 	    See the USB 2.0 spec for details about those binary data formats.
 	    You'll need to convert most multibyte values from little endian
 	    format to your native host byte order, although a few of the
 	    fields in the device descriptor (both of the BCD-encoded fields,
 	    and the vendor and product IDs) will be byteswapped for you.
 	    Note that configuration descriptors include descriptors for
 	    interfaces, altsettings, endpoints, and maybe additional
 	    class descriptors.
 	    </para>
 	    <para><emphasis>Perform USB operations</emphasis> using 
 	    <emphasis>ioctl()</emphasis> requests to make endpoint I/O
 	    requests (synchronously or asynchronously) or manage
 	    the device.
 	    These requests need the CAP_SYS_RAWIO capability,
 	    as well as filesystem access permissions.
 	    Only one ioctl request can be made on one of these
 	    device files at a time.
 	    This means that if you are synchronously reading an endpoint
 	    from one thread, you won't be able to write to a different
 	    endpoint from another thread until the read completes.
 	    This works for <emphasis>half duplex</emphasis> protocols,
 	    but otherwise you'd use asynchronous i/o requests. 
 	    </para>
 	    </sect1>
 	<sect1>
 	    <title>Life Cycle of User Mode Drivers</title>
 	    <para>Such a driver first needs to find a device file
 	    for a device it knows how to handle.
 	    Maybe it was told about it because a
 	    <filename>/sbin/hotplug</filename> event handling agent
 	    chose that driver to handle the new device.
 	    Or maybe it's an application that scans all the
 	    /proc/bus/usb device files, and ignores most devices.
 	    In either case, it should <function>read()</function> all
 	    the descriptors from the device file,
 	    and check them against what it knows how to handle.
 	    It might just reject everything except a particular
 	    vendor and product ID, or need a more complex policy.
 	    </para>
 	    <para>Never assume there will only be one such device
 	    on the system at a time!
 	    If your code can't handle more than one device at
 	    a time, at least detect when there's more than one, and
 	    have your users choose which device to use.
 	    </para>
 	    <para>Once your user mode driver knows what device to use,
 	    it interacts with it in either of two styles.
 	    The simple style is to make only control requests; some
 	    devices don't need more complex interactions than those.
 	    (An example might be software using vendor-specific control
 	    requests for some initialization or configuration tasks,
 	    with a kernel driver for the rest.)
 	    </para>
 	    <para>More likely, you need a more complex style driver:
 	    one using non-control endpoints, reading or writing data
 	    and claiming exclusive use of an interface.
 	    <emphasis>Bulk</emphasis> transfers are easiest to use,
 	    but only their sibling <emphasis>interrupt</emphasis> transfers 
 	    work with low speed devices.
 	    Both interrupt and <emphasis>isochronous</emphasis> transfers
 	    offer service guarantees because their bandwidth is reserved.
 	    Such "periodic" transfers are awkward to use through usbfs,
 	    unless you're using the asynchronous calls.  However, interrupt
 	    transfers can also be used in a synchronous "one shot" style.
 	    </para>
 	    <para>Your user-mode driver should never need to worry
 	    about cleaning up request state when the device is
 	    disconnected, although it should close its open file
 	    descriptors as soon as it starts seeing the ENODEV
 	    errors.
 	    </para>
 	    </sect1>
 	<sect1><title>The ioctl() Requests</title>
 	    <para>To use these ioctls, you need to include the following
 	    headers in your userspace program:
 <programlisting>#include &lt;linux/usb.h&gt;
 #include &lt;linux/usbdevice_fs.h&gt;
 #include &lt;asm/byteorder.h&gt;</programlisting>
 	    The standard USB device model requests, from "Chapter 9" of
 	    the USB 2.0 specification, are automatically included from
 	    the <filename>&lt;linux/usb_ch9.h&gt;</filename> header.
 	    </para>
 	    <para>Unless noted otherwise, the ioctl requests
 	    described here will
 	    update the modification time on the usbfs file to which
 	    they are applied (unless they fail).
 	    A return of zero indicates success; otherwise, a
 	    standard USB error code is returned.  (These are
 	    documented in
 	    <filename>Documentation/usb/error-codes.txt</filename>
 	    in your kernel sources.)
 	    </para>
 	    <para>Each of these files multiplexes access to several
 	    I/O streams, one per endpoint.
 	    Each device has one control endpoint (endpoint zero)
 	    which supports a limited RPC style RPC access.
 	    Devices are configured
 	    by khubd (in the kernel) setting a device-wide
 	    <emphasis>configuration</emphasis> that affects things
 	    like power consumption and basic functionality.
 	    The endpoints are part of USB <emphasis>interfaces</emphasis>,
 	    which may have <emphasis>altsettings</emphasis>
 	    affecting things like which endpoints are available.
 	    Many devices only have a single configuration and interface,
 	    so drivers for them will ignore configurations and altsettings.
 	    </para>
 	    <sect2>
 		<title>Management/Status Requests</title>
 		<para>A number of usbfs requests don't deal very directly
 		with device I/O.
 		They mostly relate to device management and status.
 		These are all synchronous requests.
 		</para>
 		<variablelist>
 		<varlistentry><term>USBDEVFS_CLAIMINTERFACE</term>
 		    <listitem><para>This is used to force usbfs to
 		    claim a specific interface,
 		    which has not previously been claimed by usbfs or any other
 		    kernel driver.
 		    The ioctl parameter is an integer holding the number of
 		    the interface (bInterfaceNumber from descriptor).
 		    </para><para>
 		    Note that if your driver doesn't claim an interface
 		    before trying to use one of its endpoints, and no
 		    other driver has bound to it, then the interface is
 		    automatically claimed by usbfs.
 		    </para><para>
 		    This claim will be released by a RELEASEINTERFACE ioctl,
 		    or by closing the file descriptor.
 		    File modification time is not updated by this request.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_CONNECTINFO</term>
 		    <listitem><para>Says whether the device is lowspeed.
 		    The ioctl parameter points to a structure like this:
 <programlisting>struct usbdevfs_connectinfo {
        unsigned int   devnum;
        unsigned char  slow;
 }; </programlisting>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    <emphasis>You can't tell whether a "not slow"
 		    device is connected at high speed (480 MBit/sec)
 		    or just full speed (12 MBit/sec).</emphasis>
 		    You should know the devnum value already,
 		    it's the DDD value of the device file name.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_GETDRIVER</term>
 		    <listitem><para>Returns the name of the kernel driver
 		    bound to a given interface (a string).  Parameter
 		    is a pointer to this structure, which is modified:
 <programlisting>struct usbdevfs_getdriver {
        unsigned int  interface;
        char          driver[USBDEVFS_MAXDRIVERNAME + 1];
 };</programlisting>
 		    File modification time is not updated by this request.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_IOCTL</term>
 		    <listitem><para>Passes a request from userspace through
 		    to a kernel driver that has an ioctl entry in the
 		    <emphasis>struct usb_driver</emphasis> it registered.
 <programlisting>struct usbdevfs_ioctl {
        int     ifno;
        int     ioctl_code;
        void    *data;
 };
 /* user mode call looks like this.
 * 'request' becomes the driver->ioctl() 'code' parameter.
 * the size of 'param' is encoded in 'request', and that data
 * is copied to or from the driver->ioctl() 'buf' parameter.
 */
 static int
 usbdev_ioctl (int fd, int ifno, unsigned request, void *param)
 {
        struct usbdevfs_ioctl	wrapper;
        wrapper.ifno = ifno;
        wrapper.ioctl_code = request;
        wrapper.data = param;
        return ioctl (fd, USBDEVFS_IOCTL, &amp;wrapper);
 } </programlisting>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    This request lets kernel drivers talk to user mode code
 		    through filesystem operations even when they don't create
 		    a charactor or block special device.
 		    It's also been used to do things like ask devices what
 		    device special file should be used.
 		    Two pre-defined ioctls are used
 		    to disconnect and reconnect kernel drivers, so
 		    that user mode code can completely manage binding
 		    and configuration of devices.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_RELEASEINTERFACE</term>
 		    <listitem><para>This is used to release the claim usbfs
 		    made on interface, either implicitly or because of a
 		    USBDEVFS_CLAIMINTERFACE call, before the file
 		    descriptor is closed.
 		    The ioctl parameter is an integer holding the number of
 		    the interface (bInterfaceNumber from descriptor);
 		    File modification time is not updated by this request.
 		    </para><warning><para>
 		    <emphasis>No security check is made to ensure
 		    that the task which made the claim is the one
 		    which is releasing it.
 		    This means that user mode driver may interfere
 		    other ones.  </emphasis>
 		    </para></warning></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_RESETEP</term>
 		    <listitem><para>Resets the data toggle value for an endpoint
 		    (bulk or interrupt) to DATA0.
 		    The ioctl parameter is an integer endpoint number
 		    (1 to 15, as identified in the endpoint descriptor),
 		    with USB_DIR_IN added if the device's endpoint sends
 		    data to the host.
 		    </para><warning><para>
 		    <emphasis>Avoid using this request.
 		    It should probably be removed.</emphasis>
 		    Using it typically means the device and driver will lose
 		    toggle synchronization.  If you really lost synchronization,
 		    you likely need to completely handshake with the device,
 		    using a request like CLEAR_HALT
 		    or SET_INTERFACE.
 		    </para></warning></listitem></varlistentry>
 		</variablelist>
 		</sect2>
 	    <sect2>
 		<title>Synchronous I/O Support</title>
 		<para>Synchronous requests involve the kernel blocking
-		until until the user mode request completes, either by
+		until the user mode request completes, either by
 		finishing successfully or by reporting an error.
 		In most cases this is the simplest way to use usbfs,
 		although as noted above it does prevent performing I/O
 		to more than one endpoint at a time.
 		</para>
 		<variablelist>
 		<varlistentry><term>USBDEVFS_BULK</term>
 		    <listitem><para>Issues a bulk read or write request to the
 		    device.
 		    The ioctl parameter is a pointer to this structure:
 <programlisting>struct usbdevfs_bulktransfer {
        unsigned int  ep;
        unsigned int  len;
        unsigned int  timeout; /* in milliseconds */
        void          *data;
 };</programlisting>
 		    </para><para>The "ep" value identifies a
 		    bulk endpoint number (1 to 15, as identified in an endpoint
 		    descriptor),
 		    masked with USB_DIR_IN when referring to an endpoint which
 		    sends data to the host from the device.
 		    The length of the data buffer is identified by "len";
 		    Recent kernels support requests up to about 128KBytes.
 		    <emphasis>FIXME say how read length is returned,
 		    and how short reads are handled.</emphasis>.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_CLEAR_HALT</term>
 		    <listitem><para>Clears endpoint halt (stall) and
 		    resets the endpoint toggle.  This is only
 		    meaningful for bulk or interrupt endpoints.
 		    The ioctl parameter is an integer endpoint number
 		    (1 to 15, as identified in an endpoint descriptor),
 		    masked with USB_DIR_IN when referring to an endpoint which
 		    sends data to the host from the device.
 		    </para><para>
 		    Use this on bulk or interrupt endpoints which have
 		    stalled, returning <emphasis>-EPIPE</emphasis> status
 		    to a data transfer request.
 		    Do not issue the control request directly, since
 		    that could invalidate the host's record of the
 		    data toggle.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_CONTROL</term>
 		    <listitem><para>Issues a control request to the device.
 		    The ioctl parameter points to a structure like this:
 <programlisting>struct usbdevfs_ctrltransfer {
        __u8   bRequestType;
        __u8   bRequest;
        __u16  wValue;
        __u16  wIndex;
        __u16  wLength;
        __u32  timeout;  /* in milliseconds */
        void   *data;
 };</programlisting>
 		    </para><para>
 		    The first eight bytes of this structure are the contents
 		    of the SETUP packet to be sent to the device; see the
 		    USB 2.0 specification for details.
 		    The bRequestType value is composed by combining a
 		    USB_TYPE_* value, a USB_DIR_* value, and a
 		    USB_RECIP_* value (from
 		    <emphasis>&lt;linux/usb.h&gt;</emphasis>).
 		    If wLength is nonzero, it describes the length of the data
 		    buffer, which is either written to the device
 		    (USB_DIR_OUT) or read from the device (USB_DIR_IN).
 		    </para><para>
 		    At this writing, you can't transfer more than 4 KBytes
 		    of data to or from a device; usbfs has a limit, and
 		    some host controller drivers have a limit.
 		    (That's not usually a problem.)
 		    <emphasis>Also</emphasis> there's no way to say it's
 		    not OK to get a short read back from the device.
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_RESET</term>
 		    <listitem><para>Does a USB level device reset.
 		    The ioctl parameter is ignored.
 		    After the reset, this rebinds all device interfaces.
 		    File modification time is not updated by this request.
 		    </para><warning><para>
 		    <emphasis>Avoid using this call</emphasis>
 		    until some usbcore bugs get fixed,
 		    since it does not fully synchronize device, interface,
 		    and driver (not just usbfs) state.
 		    </para></warning></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_SETINTERFACE</term>
 		    <listitem><para>Sets the alternate setting for an
 		    interface.  The ioctl parameter is a pointer to a
 		    structure like this:
 <programlisting>struct usbdevfs_setinterface {
        unsigned int  interface;
        unsigned int  altsetting;
 }; </programlisting>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    Those struct members are from some interface descriptor
 		    applying to the current configuration.
 		    The interface number is the bInterfaceNumber value, and
 		    the altsetting number is the bAlternateSetting value.
 		    (This resets each endpoint in the interface.)
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_SETCONFIGURATION</term>
 		    <listitem><para>Issues the
 		    <function>usb_set_configuration</function> call
 		    for the device.
 		    The parameter is an integer holding the number of
 		    a configuration (bConfigurationValue from descriptor).
 		    File modification time is not updated by this request.
 		    </para><warning><para>
 		    <emphasis>Avoid using this call</emphasis>
 		    until some usbcore bugs get fixed,
 		    since it does not fully synchronize device, interface,
 		    and driver (not just usbfs) state.
 		    </para></warning></listitem></varlistentry>
 		</variablelist>
 	    </sect2>
 	    <sect2>
 		<title>Asynchronous I/O Support</title>
 		<para>As mentioned above, there are situations where it may be
 		important to initiate concurrent operations from user mode code.
 		This is particularly important for periodic transfers
 		(interrupt and isochronous), but it can be used for other
 		kinds of USB requests too.
 		In such cases, the asynchronous requests described here
 		are essential.  Rather than submitting one request and having
 		the kernel block until it completes, the blocking is separate.
 		</para>
 		<para>These requests are packaged into a structure that
 		resembles the URB used by kernel device drivers.
 		(No POSIX Async I/O support here, sorry.)
 		It identifies the endpoint type (USBDEVFS_URB_TYPE_*),
 		endpoint (number, masked with USB_DIR_IN as appropriate),
 		buffer and length, and a user "context" value serving to
 		uniquely identify each request.
 		(It's usually a pointer to per-request data.)
 		Flags can modify requests (not as many as supported for
 		kernel drivers).
 		</para>
 		<para>Each request can specify a realtime signal number
 		(between SIGRTMIN and SIGRTMAX, inclusive) to request a
 		signal be sent when the request completes.
 		</para>
 		<para>When usbfs returns these urbs, the status value
 		is updated, and the buffer may have been modified.
 		Except for isochronous transfers, the actual_length is
 		updated to say how many bytes were transferred; if the
 		USBDEVFS_URB_DISABLE_SPD flag is set
 		("short packets are not OK"), if fewer bytes were read
 		than were requested then you get an error report.
 		</para>
 <programlisting>struct usbdevfs_iso_packet_desc {
        unsigned int                     length;
        unsigned int                     actual_length;
        unsigned int                     status;
 };
 struct usbdevfs_urb {
        unsigned char                    type;
        unsigned char                    endpoint;
        int                              status;
        unsigned int                     flags;
        void                             *buffer;
        int                              buffer_length;
        int                              actual_length;
        int                              start_frame;
        int                              number_of_packets;
        int                              error_count;
        unsigned int                     signr;
        void                             *usercontext;
        struct usbdevfs_iso_packet_desc  iso_frame_desc[];
 };</programlisting>
 		<para> For these asynchronous requests, the file modification
 		time reflects when the request was initiated.
 		This contrasts with their use with the synchronous requests,
 		where it reflects when requests complete.
 		</para>
 		<variablelist>
 		<varlistentry><term>USBDEVFS_DISCARDURB</term>
 		    <listitem><para>
 		    <emphasis>TBS</emphasis>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_DISCSIGNAL</term>
 		    <listitem><para>
 		    <emphasis>TBS</emphasis>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_REAPURB</term>
 		    <listitem><para>
 		    <emphasis>TBS</emphasis>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_REAPURBNDELAY</term>
 		    <listitem><para>
 		    <emphasis>TBS</emphasis>
 		    File modification time is not updated by this request.
 		    </para><para>
 		    </para></listitem></varlistentry>
 		<varlistentry><term>USBDEVFS_SUBMITURB</term>
 		    <listitem><para>
 		    <emphasis>TBS</emphasis>
 		    </para><para>
 		    </para></listitem></varlistentry>
 		</variablelist>
 	    </sect2>
 	</sect1>
    </chapter>
 </book>
 <!-- vim:syntax=sgml:sw=4
 -->
 What is RCU?
 RCU is a synchronization mechanism that was added to the Linux kernel
 during the 2.5 development effort that is optimized for read-mostly
 situations.  Although RCU is actually quite simple once you understand it,
 getting there can sometimes be a challenge.  Part of the problem is that
 most of the past descriptions of RCU have been written with the mistaken
 assumption that there is "one true way" to describe RCU.  Instead,
 the experience has been that different people must take different paths
 to arrive at an understanding of RCU.  This document provides several
 different paths, as follows:
 1.	RCU OVERVIEW
 2.	WHAT IS RCU'S CORE API?
 3.	WHAT ARE SOME EXAMPLE USES OF CORE RCU API?
 4.	WHAT IF MY UPDATING THREAD CANNOT BLOCK?
 5.	WHAT ARE SOME SIMPLE IMPLEMENTATIONS OF RCU?
 6.	ANALOGY WITH READER-WRITER LOCKING
 7.	FULL LIST OF RCU APIs
 8.	ANSWERS TO QUICK QUIZZES
 People who prefer starting with a conceptual overview should focus on
 Section 1, though most readers will profit by reading this section at
 some point.  People who prefer to start with an API that they can then
 experiment with should focus on Section 2.  People who prefer to start
 with example uses should focus on Sections 3 and 4.  People who need to
 understand the RCU implementation should focus on Section 5, then dive
 into the kernel source code.  People who reason best by analogy should
 focus on Section 6.  Section 7 serves as an index to the docbook API
 documentation, and Section 8 is the traditional answer key.
 So, start with the section that makes the most sense to you and your
 preferred method of learning.  If you need to know everything about
 everything, feel free to read the whole thing -- but if you are really
 that type of person, you have perused the source code and will therefore
 never need this document anyway.  ;-)
 1.  RCU OVERVIEW
 The basic idea behind RCU is to split updates into "removal" and
 "reclamation" phases.  The removal phase removes references to data items
 within a data structure (possibly by replacing them with references to
 new versions of these data items), and can run concurrently with readers.
 The reason that it is safe to run the removal phase concurrently with
 readers is the semantics of modern CPUs guarantee that readers will see
 either the old or the new version of the data structure rather than a
 partially updated reference.  The reclamation phase does the work of reclaiming
 (e.g., freeing) the data items removed from the data structure during the
 removal phase.  Because reclaiming data items can disrupt any readers
 concurrently referencing those data items, the reclamation phase must
 not start until readers no longer hold references to those data items.
 Splitting the update into removal and reclamation phases permits the
 updater to perform the removal phase immediately, and to defer the
 reclamation phase until all readers active during the removal phase have
 completed, either by blocking until they finish or by registering a
 callback that is invoked after they finish.  Only readers that are active
 during the removal phase need be considered, because any reader starting
 after the removal phase will be unable to gain a reference to the removed
 data items, and therefore cannot be disrupted by the reclamation phase.
 So the typical RCU update sequence goes something like the following:
 a.	Remove pointers to a data structure, so that subsequent
 	readers cannot gain a reference to it.
 b.	Wait for all previous readers to complete their RCU read-side
 	critical sections.
 c.	At this point, there cannot be any readers who hold references
 	to the data structure, so it now may safely be reclaimed
 	(e.g., kfree()d).
 Step (b) above is the key idea underlying RCU's deferred destruction.
 The ability to wait until all readers are done allows RCU readers to
 use much lighter-weight synchronization, in some cases, absolutely no
 synchronization at all.  In contrast, in more conventional lock-based
 schemes, readers must use heavy-weight synchronization in order to
 prevent an updater from deleting the data structure out from under them.
 This is because lock-based updaters typically update data items in place,
 and must therefore exclude readers.  In contrast, RCU-based updaters
 typically take advantage of the fact that writes to single aligned
 pointers are atomic on modern CPUs, allowing atomic insertion, removal,
 and replacement of data items in a linked structure without disrupting
 readers.  Concurrent RCU readers can then continue accessing the old
 versions, and can dispense with the atomic operations, memory barriers,
 and communications cache misses that are so expensive on present-day
 SMP computer systems, even in absence of lock contention.
 In the three-step procedure shown above, the updater is performing both
 the removal and the reclamation step, but it is often helpful for an
 entirely different thread to do the reclamation, as is in fact the case
 in the Linux kernel's directory-entry cache (dcache).  Even if the same
 thread performs both the update step (step (a) above) and the reclamation
 step (step (c) above), it is often helpful to think of them separately.
 For example, RCU readers and updaters need not communicate at all,
 but RCU provides implicit low-overhead communication between readers
 and reclaimers, namely, in step (b) above.
 So how the heck can a reclaimer tell when a reader is done, given
 that readers are not doing any sort of synchronization operations???
 Read on to learn about how RCU's API makes this easy.
 2.  WHAT IS RCU'S CORE API?
 The core RCU API is quite small:
 a.	rcu_read_lock()
 b.	rcu_read_unlock()
 c.	synchronize_rcu() / call_rcu()
 d.	rcu_assign_pointer()
 e.	rcu_dereference()
 There are many other members of the RCU API, but the rest can be
 expressed in terms of these five, though most implementations instead
 express synchronize_rcu() in terms of the call_rcu() callback API.
 The five core RCU APIs are described below, the other 18 will be enumerated
 later.  See the kernel docbook documentation for more info, or look directly
 at the function header comments.
 rcu_read_lock()
 	void rcu_read_lock(void);
 	Used by a reader to inform the reclaimer that the reader is
 	entering an RCU read-side critical section.  It is illegal
 	to block while in an RCU read-side critical section, though
 	kernels built with CONFIG_PREEMPT_RCU can preempt RCU read-side
 	critical sections.  Any RCU-protected data structure accessed
 	during an RCU read-side critical section is guaranteed to remain
 	unreclaimed for the full duration of that critical section.
 	Reference counts may be used in conjunction with RCU to maintain
 	longer-term references to data structures.
 rcu_read_unlock()
 	void rcu_read_unlock(void);
 	Used by a reader to inform the reclaimer that the reader is
 	exiting an RCU read-side critical section.  Note that RCU
 	read-side critical sections may be nested and/or overlapping.
 synchronize_rcu()
 	void synchronize_rcu(void);
 	Marks the end of updater code and the beginning of reclaimer
 	code.  It does this by blocking until all pre-existing RCU
 	read-side critical sections on all CPUs have completed.
 	Note that synchronize_rcu() will -not- necessarily wait for
 	any subsequent RCU read-side critical sections to complete.
 	For example, consider the following sequence of events:
 	         CPU 0                  CPU 1                 CPU 2
 	     ----------------- ------------------------- ---------------
 	 1.  rcu_read_lock()
 	 2.                    enters synchronize_rcu()
 	 3.                                               rcu_read_lock()
 	 4.  rcu_read_unlock()
 	 5.                     exits synchronize_rcu()
 	 6.                                              rcu_read_unlock()
 	To reiterate, synchronize_rcu() waits only for ongoing RCU
 	read-side critical sections to complete, not necessarily for
 	any that begin after synchronize_rcu() is invoked.
 	Of course, synchronize_rcu() does not necessarily return
 	-immediately- after the last pre-existing RCU read-side critical
 	section completes.  For one thing, there might well be scheduling
 	delays.  For another thing, many RCU implementations process
 	requests in batches in order to improve efficiencies, which can
 	further delay synchronize_rcu().
 	Since synchronize_rcu() is the API that must figure out when
 	readers are done, its implementation is key to RCU.  For RCU
 	to be useful in all but the most read-intensive situations,
 	synchronize_rcu()'s overhead must also be quite small.
 	The call_rcu() API is a callback form of synchronize_rcu(),
 	and is described in more detail in a later section.  Instead of
 	blocking, it registers a function and argument which are invoked
 	after all ongoing RCU read-side critical sections have completed.
 	This callback variant is particularly useful in situations where
 	it is illegal to block or where update-side performance is
 	critically important.
 	However, the call_rcu() API should not be used lightly, as use
 	of the synchronize_rcu() API generally results in simpler code.
 	In addition, the synchronize_rcu() API has the nice property
 	of automatically limiting update rate should grace periods
 	be delayed.  This property results in system resilience in face
 	of denial-of-service attacks.  Code using call_rcu() should limit
 	update rate in order to gain this same sort of resilience.  See
 	checklist.txt for some approaches to limiting the update rate.
 rcu_assign_pointer()
 	typeof(p) rcu_assign_pointer(p, typeof(p) v);
 	Yes, rcu_assign_pointer() -is- implemented as a macro, though it
 	would be cool to be able to declare a function in this manner.
 	(Compiler experts will no doubt disagree.)
 	The updater uses this function to assign a new value to an
 	RCU-protected pointer, in order to safely communicate the change
 	in value from the updater to the reader.  This function returns
 	the new value, and also executes any memory-barrier instructions
 	required for a given CPU architecture.
 	Perhaps just as important, it serves to document (1) which
 	pointers are protected by RCU and (2) the point at which a
 	given structure becomes accessible to other CPUs.  That said,
 	rcu_assign_pointer() is most frequently used indirectly, via
 	the _rcu list-manipulation primitives such as list_add_rcu().
 rcu_dereference()
 	typeof(p) rcu_dereference(p);
 	Like rcu_assign_pointer(), rcu_dereference() must be implemented
 	as a macro.
 	The reader uses rcu_dereference() to fetch an RCU-protected
 	pointer, which returns a value that may then be safely
 	dereferenced.  Note that rcu_deference() does not actually
 	dereference the pointer, instead, it protects the pointer for
 	later dereferencing.  It also executes any needed memory-barrier
 	instructions for a given CPU architecture.  Currently, only Alpha
 	needs memory barriers within rcu_dereference() -- on other CPUs,
 	it compiles to nothing, not even a compiler directive.
 	Common coding practice uses rcu_dereference() to copy an
 	RCU-protected pointer to a local variable, then dereferences
 	this local variable, for example as follows:
 		p = rcu_dereference(head.next);
 		return p->data;
 	However, in this case, one could just as easily combine these
 	into one statement:
 		return rcu_dereference(head.next)->data;
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
 	ugly and incur unnecessary overhead on Alpha CPUs.
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
 	For example, the following is -not- legal:
 		rcu_read_lock();
 		p = rcu_dereference(head.next);
 		rcu_read_unlock();
 		x = p->address;
 		rcu_read_lock();
 		y = p->data;
 		rcu_read_unlock();
 	Holding a reference from one RCU read-side critical section
 	to another is just as illegal as holding a reference from
 	one lock-based critical section to another!  Similarly,
 	using a reference outside of the critical section in which
 	it was acquired is just as illegal as doing so with normal
 	locking.
 	As with rcu_assign_pointer(), an important function of
 	rcu_dereference() is to document which pointers are protected by
 	RCU, in particular, flagging a pointer that is subject to changing
 	at any time, including immediately after the rcu_dereference().
 	And, again like rcu_assign_pointer(), rcu_dereference() is
 	typically used indirectly, via the _rcu list-manipulation
 	primitives, such as list_for_each_entry_rcu().
 The following diagram shows how each API communicates among the
 reader, updater, and reclaimer.
 	    rcu_assign_pointer()
 	    			    +--------+
 	    +---------------------->| reader |---------+
 	    |                       +--------+         |
 	    |                           |              |
 	    |                           |              | Protect:
 	    |                           |              | rcu_read_lock()
 	    |                           |              | rcu_read_unlock()
 	    |        rcu_dereference()  |              |
       +---------+                      |              |
       | updater |<---------------------+              |
       +---------+                                     V
 	    |                                    +-----------+
 	    +----------------------------------->| reclaimer |
 	    				         +-----------+
 	      Defer:
 	      synchronize_rcu() & call_rcu()
 The RCU infrastructure observes the time sequence of rcu_read_lock(),
 rcu_read_unlock(), synchronize_rcu(), and call_rcu() invocations in
 order to determine when (1) synchronize_rcu() invocations may return
 to their callers and (2) call_rcu() callbacks may be invoked.  Efficient
 implementations of the RCU infrastructure make heavy use of batching in
 order to amortize their overhead over many uses of the corresponding APIs.
 There are no fewer than three RCU mechanisms in the Linux kernel; the
 diagram above shows the first one, which is by far the most commonly used.
 The rcu_dereference() and rcu_assign_pointer() primitives are used for
 all three mechanisms, but different defer and protect primitives are
 used as follows:
 	Defer			Protect
 a.	synchronize_rcu()	rcu_read_lock() / rcu_read_unlock()
 	call_rcu()
 b.	call_rcu_bh()		rcu_read_lock_bh() / rcu_read_unlock_bh()
 c.	synchronize_sched()	preempt_disable() / preempt_enable()
 				local_irq_save() / local_irq_restore()
 				hardirq enter / hardirq exit
 				NMI enter / NMI exit
 These three mechanisms are used as follows:
 a.	RCU applied to normal data structures.
 b.	RCU applied to networking data structures that may be subjected
 	to remote denial-of-service attacks.
 c.	RCU applied to scheduler and interrupt/NMI-handler tasks.
 Again, most uses will be of (a).  The (b) and (c) cases are important
 for specialized uses, but are relatively uncommon.
 3.  WHAT ARE SOME EXAMPLE USES OF CORE RCU API?
 This section shows a simple use of the core RCU API to protect a
 global pointer to a dynamically allocated structure.  More-typical
 uses of RCU may be found in listRCU.txt, arrayRCU.txt, and NMI-RCU.txt.
 	struct foo {
 		int a;
 		char b;
 		long c;
 	};
 	DEFINE_SPINLOCK(foo_mutex);
 	struct foo *gbl_foo;
 	/*
 	 * Create a new struct foo that is the same as the one currently
 	 * pointed to by gbl_foo, except that field "a" is replaced
 	 * with "new_a".  Points gbl_foo to the new structure, and
 	 * frees up the old structure after a grace period.
 	 *
 	 * Uses rcu_assign_pointer() to ensure that concurrent readers
 	 * see the initialized version of the new structure.
 	 *
 	 * Uses synchronize_rcu() to ensure that any readers that might
 	 * have references to the old structure complete before freeing
 	 * the old structure.
 	 */
 	void foo_update_a(int new_a)
 	{
 		struct foo *new_fp;
 		struct foo *old_fp;
 		new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
 		spin_lock(&foo_mutex);
 		old_fp = gbl_foo;
 		*new_fp = *old_fp;
 		new_fp->a = new_a;
 		rcu_assign_pointer(gbl_foo, new_fp);
 		spin_unlock(&foo_mutex);
 		synchronize_rcu();
 		kfree(old_fp);
 	}
 	/*
 	 * Return the value of field "a" of the current gbl_foo
 	 * structure.  Use rcu_read_lock() and rcu_read_unlock()
 	 * to ensure that the structure does not get deleted out
 	 * from under us, and use rcu_dereference() to ensure that
 	 * we see the initialized version of the structure (important
 	 * for DEC Alpha and for people reading the code).
 	 */
 	int foo_get_a(void)
 	{
 		int retval;
 		rcu_read_lock();
 		retval = rcu_dereference(gbl_foo)->a;
 		rcu_read_unlock();
 		return retval;
 	}
 So, to sum up:
 o	Use rcu_read_lock() and rcu_read_unlock() to guard RCU
 	read-side critical sections.
 o	Within an RCU read-side critical section, use rcu_dereference()
 	to dereference RCU-protected pointers.
 o	Use some solid scheme (such as locks or semaphores) to
 	keep concurrent updates from interfering with each other.
 o	Use rcu_assign_pointer() to update an RCU-protected pointer.
 	This primitive protects concurrent readers from the updater,
 	-not- concurrent updates from each other!  You therefore still
 	need to use locking (or something similar) to keep concurrent
 	rcu_assign_pointer() primitives from interfering with each other.
 o	Use synchronize_rcu() -after- removing a data element from an
 	RCU-protected data structure, but -before- reclaiming/freeing
 	the data element, in order to wait for the completion of all
 	RCU read-side critical sections that might be referencing that
 	data item.
 See checklist.txt for additional rules to follow when using RCU.
 And again, more-typical uses of RCU may be found in listRCU.txt,
 arrayRCU.txt, and NMI-RCU.txt.
 4.  WHAT IF MY UPDATING THREAD CANNOT BLOCK?
 In the example above, foo_update_a() blocks until a grace period elapses.
 This is quite simple, but in some cases one cannot afford to wait so
 long -- there might be other high-priority work to be done.
 In such cases, one uses call_rcu() rather than synchronize_rcu().
 The call_rcu() API is as follows:
 	void call_rcu(struct rcu_head * head,
 		      void (*func)(struct rcu_head *head));
 This function invokes func(head) after a grace period has elapsed.
 This invocation might happen from either softirq or process context,
 so the function is not permitted to block.  The foo struct needs to
 have an rcu_head structure added, perhaps as follows:
 	struct foo {
 		int a;
 		char b;
 		long c;
 		struct rcu_head rcu;
 	};
 The foo_update_a() function might then be written as follows:
 	/*
 	 * Create a new struct foo that is the same as the one currently
 	 * pointed to by gbl_foo, except that field "a" is replaced
 	 * with "new_a".  Points gbl_foo to the new structure, and
 	 * frees up the old structure after a grace period.
 	 *
 	 * Uses rcu_assign_pointer() to ensure that concurrent readers
 	 * see the initialized version of the new structure.
 	 *
 	 * Uses call_rcu() to ensure that any readers that might have
 	 * references to the old structure complete before freeing the
 	 * old structure.
 	 */
 	void foo_update_a(int new_a)
 	{
 		struct foo *new_fp;
 		struct foo *old_fp;
 		new_fp = kmalloc(sizeof(*new_fp), GFP_KERNEL);
 		spin_lock(&foo_mutex);
 		old_fp = gbl_foo;
 		*new_fp = *old_fp;
 		new_fp->a = new_a;
 		rcu_assign_pointer(gbl_foo, new_fp);
 		spin_unlock(&foo_mutex);
 		call_rcu(&old_fp->rcu, foo_reclaim);
 	}
 The foo_reclaim() function might appear as follows:
 	void foo_reclaim(struct rcu_head *rp)
 	{
 		struct foo *fp = container_of(rp, struct foo, rcu);
 		kfree(fp);
 	}
 The container_of() primitive is a macro that, given a pointer into a
 struct, the type of the struct, and the pointed-to field within the
 struct, returns a pointer to the beginning of the struct.
 The use of call_rcu() permits the caller of foo_update_a() to
 immediately regain control, without needing to worry further about the
 old version of the newly updated element.  It also clearly shows the
 RCU distinction between updater, namely foo_update_a(), and reclaimer,
 namely foo_reclaim().
 The summary of advice is the same as for the previous section, except
 that we are now using call_rcu() rather than synchronize_rcu():
 o	Use call_rcu() -after- removing a data element from an
 	RCU-protected data structure in order to register a callback
 	function that will be invoked after the completion of all RCU
 	read-side critical sections that might be referencing that
 	data item.
 Again, see checklist.txt for additional rules governing the use of RCU.
 5.  WHAT ARE SOME SIMPLE IMPLEMENTATIONS OF RCU?
 One of the nice things about RCU is that it has extremely simple "toy"
 implementations that are a good first step towards understanding the
 production-quality implementations in the Linux kernel.  This section
 presents two such "toy" implementations of RCU, one that is implemented
 in terms of familiar locking primitives, and another that more closely
 resembles "classic" RCU.  Both are way too simple for real-world use,
 lacking both functionality and performance.  However, they are useful
 in getting a feel for how RCU works.  See kernel/rcupdate.c for a
 production-quality implementation, and see:
 	http://www.rdrop.com/users/paulmck/RCU
 for papers describing the Linux kernel RCU implementation.  The OLS'01
 and OLS'02 papers are a good introduction, and the dissertation provides
 more details on the current implementation as of early 2004.
 5A.  "TOY" IMPLEMENTATION #1: LOCKING
 This section presents a "toy" RCU implementation that is based on
 familiar locking primitives.  Its overhead makes it a non-starter for
 real-life use, as does its lack of scalability.  It is also unsuitable
 for realtime use, since it allows scheduling latency to "bleed" from
 one read-side critical section to another.
 However, it is probably the easiest implementation to relate to, so is
 a good starting point.
 It is extremely simple:
 	static DEFINE_RWLOCK(rcu_gp_mutex);
 	void rcu_read_lock(void)
 	{
 		read_lock(&rcu_gp_mutex);
 	}
 	void rcu_read_unlock(void)
 	{
 		read_unlock(&rcu_gp_mutex);
 	}
 	void synchronize_rcu(void)
 	{
 		write_lock(&rcu_gp_mutex);
 		write_unlock(&rcu_gp_mutex);
 	}
 [You can ignore rcu_assign_pointer() and rcu_dereference() without
 missing much.  But here they are anyway.  And whatever you do, don't
 forget about them when submitting patches making use of RCU!]
 	#define rcu_assign_pointer(p, v)	({ \
 							smp_wmb(); \
 							(p) = (v); \
 						})
 	#define rcu_dereference(p)     ({ \
 					typeof(p) _________p1 = p; \
 					smp_read_barrier_depends(); \
 					(_________p1); \
 					})
 The rcu_read_lock() and rcu_read_unlock() primitive read-acquire
 and release a global reader-writer lock.  The synchronize_rcu()
 primitive write-acquires this same lock, then immediately releases
 it.  This means that once synchronize_rcu() exits, all RCU read-side
 critical sections that were in progress before synchronize_rcu() was
 called are guaranteed to have completed -- there is no way that
 synchronize_rcu() would have been able to write-acquire the lock
 otherwise.
 It is possible to nest rcu_read_lock(), since reader-writer locks may
 be recursively acquired.  Note also that rcu_read_lock() is immune
 from deadlock (an important property of RCU).  The reason for this is
 that the only thing that can block rcu_read_lock() is a synchronize_rcu().
 But synchronize_rcu() does not acquire any locks while holding rcu_gp_mutex,
 so there can be no deadlock cycle.
 Quick Quiz #1:	Why is this argument naive?  How could a deadlock
 		occur when using this algorithm in a real-world Linux
 		kernel?  How could this deadlock be avoided?
 5B.  "TOY" EXAMPLE #2: CLASSIC RCU
 This section presents a "toy" RCU implementation that is based on
 "classic RCU".  It is also short on performance (but only for updates) and
 on features such as hotplug CPU and the ability to run in CONFIG_PREEMPT
 kernels.  The definitions of rcu_dereference() and rcu_assign_pointer()
 are the same as those shown in the preceding section, so they are omitted.
 	void rcu_read_lock(void) { }
 	void rcu_read_unlock(void) { }
 	void synchronize_rcu(void)
 	{
 		int cpu;
 		for_each_possible_cpu(cpu)
 			run_on(cpu);
 	}
 Note that rcu_read_lock() and rcu_read_unlock() do absolutely nothing.
 This is the great strength of classic RCU in a non-preemptive kernel:
 read-side overhead is precisely zero, at least on non-Alpha CPUs.
 And there is absolutely no way that rcu_read_lock() can possibly
 participate in a deadlock cycle!
 The implementation of synchronize_rcu() simply schedules itself on each
 CPU in turn.  The run_on() primitive can be implemented straightforwardly
 in terms of the sched_setaffinity() primitive.  Of course, a somewhat less
 "toy" implementation would restore the affinity upon completion rather
 than just leaving all tasks running on the last CPU, but when I said
 "toy", I meant -toy-!
 So how the heck is this supposed to work???
 Remember that it is illegal to block while in an RCU read-side critical
 section.  Therefore, if a given CPU executes a context switch, we know
 that it must have completed all preceding RCU read-side critical sections.
 Once -all- CPUs have executed a context switch, then -all- preceding
 RCU read-side critical sections will have completed.
 So, suppose that we remove a data item from its structure and then invoke
 synchronize_rcu().  Once synchronize_rcu() returns, we are guaranteed
 that there are no RCU read-side critical sections holding a reference
 to that data item, so we can safely reclaim it.
 Quick Quiz #2:	Give an example where Classic RCU's read-side
 		overhead is -negative-.
 Quick Quiz #3:  If it is illegal to block in an RCU read-side
 		critical section, what the heck do you do in
 		PREEMPT_RT, where normal spinlocks can block???
 6.  ANALOGY WITH READER-WRITER LOCKING
 Although RCU can be used in many different ways, a very common use of
 RCU is analogous to reader-writer locking.  The following unified
 diff shows how closely related RCU and reader-writer locking can be.
 	@@ -13,15 +14,15 @@
 		struct list_head *lp;
 		struct el *p;
 	-	read_lock();
 	-	list_for_each_entry(p, head, lp) {
 	+	rcu_read_lock();
 	+	list_for_each_entry_rcu(p, head, lp) {
 			if (p->key == key) {
 				*result = p->data;
 	-			read_unlock();
 	+			rcu_read_unlock();
 				return 1;
 			}
 		}
 	-	read_unlock();
 	+	rcu_read_unlock();
 		return 0;
 	 }
 	@@ -29,15 +30,16 @@
 	 {
 		struct el *p;
 	-	write_lock(&listmutex);
 	+	spin_lock(&listmutex);
 		list_for_each_entry(p, head, lp) {
 			if (p->key == key) {
 	-			list_del(&p->list);
 	-			write_unlock(&listmutex);
 	+			list_del_rcu(&p->list);
 	+			spin_unlock(&listmutex);
 	+			synchronize_rcu();
 				kfree(p);
 				return 1;
 			}
 		}
 	-	write_unlock(&listmutex);
 	+	spin_unlock(&listmutex);
 		return 0;
 	 }
 Or, for those who prefer a side-by-side listing:
 1 struct el {                          1 struct el {
 2   struct list_head list;             2   struct list_head list;
 3   long key;                          3   long key;
 4   spinlock_t mutex;                  4   spinlock_t mutex;
 5   int data;                          5   int data;
 6   /* Other data fields */            6   /* Other data fields */
 7 };                                   7 };
 8 spinlock_t listmutex;                8 spinlock_t listmutex;
 9 struct el head;                      9 struct el head;
 1 int search(long key, int *result)    1 int search(long key, int *result)
 2 {                                    2 {
 3   struct list_head *lp;              3   struct list_head *lp;
 4   struct el *p;                      4   struct el *p;
 5                                      5
 6   read_lock();                       6   rcu_read_lock();
 7   list_for_each_entry(p, head, lp) { 7   list_for_each_entry_rcu(p, head, lp) {
 8     if (p->key == key) {             8     if (p->key == key) {
 9       *result = p->data;             9       *result = p->data;
 10       read_unlock();                10       rcu_read_unlock();
 11       return 1;                     11       return 1;
 12     }                               12     }
 13   }                                 13   }
 14   read_unlock();                    14   rcu_read_unlock();
 15   return 0;                         15   return 0;
 16 }                                   16 }
 1 int delete(long key)                 1 int delete(long key)
 2 {                                    2 {
 3   struct el *p;                      3   struct el *p;
 4                                      4
 5   write_lock(&listmutex);            5   spin_lock(&listmutex);
 6   list_for_each_entry(p, head, lp) { 6   list_for_each_entry(p, head, lp) {
 7     if (p->key == key) {             7     if (p->key == key) {
 8       list_del(&p->list);            8       list_del_rcu(&p->list);
 9       write_unlock(&listmutex);      9       spin_unlock(&listmutex);
                                       10       synchronize_rcu();
 10       kfree(p);                     11       kfree(p);
 11       return 1;                     12       return 1;
 12     }                               13     }
 13   }                                 14   }
 14   write_unlock(&listmutex);         15   spin_unlock(&listmutex);
 15   return 0;                         16   return 0;
 16 }                                   17 }
 Either way, the differences are quite small.  Read-side locking moves
 to rcu_read_lock() and rcu_read_unlock, update-side locking moves from
-from a reader-writer lock to a simple spinlock, and a synchronize_rcu()
+a reader-writer lock to a simple spinlock, and a synchronize_rcu()
 precedes the kfree().
 However, there is one potential catch: the read-side and update-side
 critical sections can now run concurrently.  In many cases, this will
 not be a problem, but it is necessary to check carefully regardless.
 For example, if multiple independent list updates must be seen as
 a single atomic update, converting to RCU will require special care.
 Also, the presence of synchronize_rcu() means that the RCU version of
 delete() can now block.  If this is a problem, there is a callback-based
 mechanism that never blocks, namely call_rcu(), that can be used in
 place of synchronize_rcu().
 7.  FULL LIST OF RCU APIs
 The RCU APIs are documented in docbook-format header comments in the
 Linux-kernel source code, but it helps to have a full list of the
 APIs, since there does not appear to be a way to categorize them
 in docbook.  Here is the list, by category.
 Markers for RCU read-side critical sections:
 	rcu_read_lock
 	rcu_read_unlock
 	rcu_read_lock_bh
 	rcu_read_unlock_bh
 RCU pointer/list traversal:
 	rcu_dereference
 	list_for_each_rcu		(to be deprecated in favor of
 					 list_for_each_entry_rcu)
 	list_for_each_entry_rcu
 	list_for_each_continue_rcu	(to be deprecated in favor of new
 					 list_for_each_entry_continue_rcu)
 	hlist_for_each_entry_rcu
 RCU pointer update:
 	rcu_assign_pointer
 	list_add_rcu
 	list_add_tail_rcu
 	list_del_rcu
 	list_replace_rcu
 	hlist_del_rcu
 	hlist_add_head_rcu
 RCU grace period:
 	synchronize_net
 	synchronize_sched
 	synchronize_rcu
 	call_rcu
 	call_rcu_bh
 See the comment headers in the source code (or the docbook generated
 from them) for more information.
 8.  ANSWERS TO QUICK QUIZZES
 Quick Quiz #1:	Why is this argument naive?  How could a deadlock
 		occur when using this algorithm in a real-world Linux
 		kernel?  [Referring to the lock-based "toy" RCU
 		algorithm.]
 Answer:		Consider the following sequence of events:
 		1.	CPU 0 acquires some unrelated lock, call it
 			"problematic_lock", disabling irq via
 			spin_lock_irqsave().
 		2.	CPU 1 enters synchronize_rcu(), write-acquiring
 			rcu_gp_mutex.
 		3.	CPU 0 enters rcu_read_lock(), but must wait
 			because CPU 1 holds rcu_gp_mutex.
 		4.	CPU 1 is interrupted, and the irq handler
 			attempts to acquire problematic_lock.
 		The system is now deadlocked.
 		One way to avoid this deadlock is to use an approach like
 		that of CONFIG_PREEMPT_RT, where all normal spinlocks
 		become blocking locks, and all irq handlers execute in
 		the context of special tasks.  In this case, in step 4
 		above, the irq handler would block, allowing CPU 1 to
 		release rcu_gp_mutex, avoiding the deadlock.
 		Even in the absence of deadlock, this RCU implementation
 		allows latency to "bleed" from readers to other
 		readers through synchronize_rcu().  To see this,
 		consider task A in an RCU read-side critical section
 		(thus read-holding rcu_gp_mutex), task B blocked
 		attempting to write-acquire rcu_gp_mutex, and
 		task C blocked in rcu_read_lock() attempting to
 		read_acquire rcu_gp_mutex.  Task A's RCU read-side
 		latency is holding up task C, albeit indirectly via
 		task B.
 		Realtime RCU implementations therefore use a counter-based
 		approach where tasks in RCU read-side critical sections
 		cannot be blocked by tasks executing synchronize_rcu().
 Quick Quiz #2:	Give an example where Classic RCU's read-side
 		overhead is -negative-.
 Answer:		Imagine a single-CPU system with a non-CONFIG_PREEMPT
 		kernel where a routing table is used by process-context
 		code, but can be updated by irq-context code (for example,
 		by an "ICMP REDIRECT" packet).	The usual way of handling
 		this would be to have the process-context code disable
 		interrupts while searching the routing table.  Use of
 		RCU allows such interrupt-disabling to be dispensed with.
 		Thus, without RCU, you pay the cost of disabling interrupts,
 		and with RCU you don't.
 		One can argue that the overhead of RCU in this
 		case is negative with respect to the single-CPU
 		interrupt-disabling approach.  Others might argue that
 		the overhead of RCU is merely zero, and that replacing
 		the positive overhead of the interrupt-disabling scheme
 		with the zero-overhead RCU scheme does not constitute
 		negative overhead.
 		In real life, of course, things are more complex.  But
 		even the theoretical possibility of negative overhead for
 		a synchronization primitive is a bit unexpected.  ;-)
 Quick Quiz #3:  If it is illegal to block in an RCU read-side
 		critical section, what the heck do you do in
 		PREEMPT_RT, where normal spinlocks can block???
 Answer:		Just as PREEMPT_RT permits preemption of spinlock
 		critical sections, it permits preemption of RCU
 		read-side critical sections.  It also permits
 		spinlocks blocking while in RCU read-side critical
 		sections.
 		Why the apparent inconsistency?  Because it is it
 		possible to use priority boosting to keep the RCU
 		grace periods short if need be (for example, if running
 		short of memory).  In contrast, if blocking waiting
 		for (say) network reception, there is no way to know
 		what should be boosted.  Especially given that the
 		process we need to boost might well be a human being
 		who just went out for a pizza or something.  And although
 		a computer-operated cattle prod might arouse serious
 		interest, it might also provoke serious objections.
 		Besides, how does the computer know what pizza parlor
 		the human being went to???
 ACKNOWLEDGEMENTS
 My thanks to the people who helped make this human-readable, including
 Jon Walpole, Josh Triplett, Serge Hallyn, Suzanne Wood, and Alan Stern.
 For more information, see http://www.rdrop.com/users/paulmck/RCU.
 	Notes on the Generic Block Layer Rewrite in Linux 2.5
 	=====================================================
 Notes Written on Jan 15, 2002:
 	Jens Axboe <axboe@suse.de>
 	Suparna Bhattacharya <suparna@in.ibm.com>
 Last Updated May 2, 2002
 September 2003: Updated I/O Scheduler portions
 	Nick Piggin <piggin@cyberone.com.au>
 Introduction:
 These are some notes describing some aspects of the 2.5 block layer in the
 context of the bio rewrite. The idea is to bring out some of the key
 changes and a glimpse of the rationale behind those changes.
 Please mail corrections & suggestions to suparna@in.ibm.com.
 Credits:
 ---------
 2.5 bio rewrite:
 	Jens Axboe <axboe@suse.de>
 Many aspects of the generic block layer redesign were driven by and evolved
 over discussions, prior patches and the collective experience of several
 people. See sections 8 and 9 for a list of some related references.
 The following people helped with review comments and inputs for this
 document:
 	Christoph Hellwig <hch@infradead.org>
 	Arjan van de Ven <arjanv@redhat.com>
 	Randy Dunlap <rdunlap@xenotime.net>
 	Andre Hedrick <andre@linux-ide.org>
 The following people helped with fixes/contributions to the bio patches
 while it was still work-in-progress:
 	David S. Miller <davem@redhat.com>
 Description of Contents:
 ------------------------
 1. Scope for tuning of logic to various needs
  1.1 Tuning based on device or low level driver capabilities
 	- Per-queue parameters
 	- Highmem I/O support
 	- I/O scheduler modularization
  1.2 Tuning based on high level requirements/capabilities
 	1.2.1 I/O Barriers
 	1.2.2 Request Priority/Latency
  1.3 Direct access/bypass to lower layers for diagnostics and special
      device operations
 	1.3.1 Pre-built commands
 2. New flexible and generic but minimalist i/o structure or descriptor
   (instead of using buffer heads at the i/o layer)
  2.1 Requirements/Goals addressed
  2.2 The bio struct in detail (multi-page io unit)
  2.3 Changes in the request structure
 3. Using bios
  3.1 Setup/teardown (allocation, splitting)
  3.2 Generic bio helper routines
    3.2.1 Traversing segments and completion units in a request
    3.2.2 Setting up DMA scatterlists
    3.2.3 I/O completion
    3.2.4 Implications for drivers that do not interpret bios (don't handle
 	  multiple segments)
    3.2.5 Request command tagging
  3.3 I/O submission
 4. The I/O scheduler
 5. Scalability related changes
  5.1 Granular locking: Removal of io_request_lock
  5.2 Prepare for transition to 64 bit sector_t
 6. Other Changes/Implications
  6.1 Partition re-mapping handled by the generic block layer
 7. A few tips on migration of older drivers
 8. A list of prior/related/impacted patches/ideas
 9. Other References/Discussion Threads
 ---------------------------------------------------------------------------
 Bio Notes
 --------
 Let us discuss the changes in the context of how some overall goals for the
 block layer are addressed.
 1. Scope for tuning the generic logic to satisfy various requirements
 The block layer design supports adaptable abstractions to handle common
 processing with the ability to tune the logic to an appropriate extent
 depending on the nature of the device and the requirements of the caller.
 One of the objectives of the rewrite was to increase the degree of tunability
 and to enable higher level code to utilize underlying device/driver
 capabilities to the maximum extent for better i/o performance. This is
 important especially in the light of ever improving hardware capabilities
 and application/middleware software designed to take advantage of these
 capabilities.
 1.1 Tuning based on low level device / driver capabilities
 Sophisticated devices with large built-in caches, intelligent i/o scheduling
 optimizations, high memory DMA support, etc may find some of the
 generic processing an overhead, while for less capable devices the
 generic functionality is essential for performance or correctness reasons.
 Knowledge of some of the capabilities or parameters of the device should be
 used at the generic block layer to take the right decisions on
 behalf of the driver.
 How is this achieved ?
 Tuning at a per-queue level:
 i. Per-queue limits/values exported to the generic layer by the driver
 Various parameters that the generic i/o scheduler logic uses are set at
 a per-queue level (e.g maximum request size, maximum number of segments in
 a scatter-gather list, hardsect size)
 Some parameters that were earlier available as global arrays indexed by
 major/minor are now directly associated with the queue. Some of these may
 move into the block device structure in the future. Some characteristics
 have been incorporated into a queue flags field rather than separate fields
 in themselves.  There are blk_queue_xxx functions to set the parameters,
 rather than update the fields directly
 Some new queue property settings:
 	blk_queue_bounce_limit(q, u64 dma_address)
 		Enable I/O to highmem pages, dma_address being the
 		limit. No highmem default.
 	blk_queue_max_sectors(q, max_sectors)
 		Sets two variables that limit the size of the request.
 		- The request queue's max_sectors, which is a soft size in
-		in units of 512 byte sectors, and could be dynamically varied
+		units of 512 byte sectors, and could be dynamically varied
 		by the core kernel.
 		- The request queue's max_hw_sectors, which is a hard limit
 		and reflects the maximum size request a driver can handle
 		in units of 512 byte sectors.
 		The default for both max_sectors and max_hw_sectors is
 		255. The upper limit of max_sectors is 1024.
 	blk_queue_max_phys_segments(q, max_segments)
 		Maximum physical segments you can handle in a request. 128
 		default (driver limit). (See 3.2.2)
 	blk_queue_max_hw_segments(q, max_segments)
 		Maximum dma segments the hardware can handle in a request. 128
 		default (host adapter limit, after dma remapping).
 		(See 3.2.2)
 	blk_queue_max_segment_size(q, max_seg_size)
 		Maximum size of a clustered segment, 64kB default.
 	blk_queue_hardsect_size(q, hardsect_size)
 		Lowest possible sector size that the hardware can operate
 		on, 512 bytes default.
 New queue flags:
 	QUEUE_FLAG_CLUSTER (see 3.2.2)
 	QUEUE_FLAG_QUEUED (see 3.2.4)
 ii. High-mem i/o capabilities are now considered the default
 The generic bounce buffer logic, present in 2.4, where the block layer would
 by default copyin/out i/o requests on high-memory buffers to low-memory buffers
 assuming that the driver wouldn't be able to handle it directly, has been
 changed in 2.5. The bounce logic is now applied only for memory ranges
 for which the device cannot handle i/o. A driver can specify this by
 setting the queue bounce limit for the request queue for the device
 (blk_queue_bounce_limit()). This avoids the inefficiencies of the copyin/out
 where a device is capable of handling high memory i/o.
 In order to enable high-memory i/o where the device is capable of supporting
 it, the pci dma mapping routines and associated data structures have now been
 modified to accomplish a direct page -> bus translation, without requiring
 a virtual address mapping (unlike the earlier scheme of virtual address
 -> bus translation). So this works uniformly for high-memory pages (which
 do not have a correponding kernel virtual address space mapping) and
 low-memory pages.
 Note: Please refer to DMA-mapping.txt for a discussion on PCI high mem DMA
 aspects and mapping of scatter gather lists, and support for 64 bit PCI.
 Special handling is required only for cases where i/o needs to happen on
 pages at physical memory addresses beyond what the device can support. In these
 cases, a bounce bio representing a buffer from the supported memory range
 is used for performing the i/o with copyin/copyout as needed depending on
 the type of the operation.  For example, in case of a read operation, the
 data read has to be copied to the original buffer on i/o completion, so a
 callback routine is set up to do this, while for write, the data is copied
 from the original buffer to the bounce buffer prior to issuing the
 operation. Since an original buffer may be in a high memory area that's not
 mapped in kernel virtual addr, a kmap operation may be required for
 performing the copy, and special care may be needed in the completion path
 as it may not be in irq context. Special care is also required (by way of
 GFP flags) when allocating bounce buffers, to avoid certain highmem
 deadlock possibilities.
 It is also possible that a bounce buffer may be allocated from high-memory
 area that's not mapped in kernel virtual addr, but within the range that the
 device can use directly; so the bounce page may need to be kmapped during
 copy operations. [Note: This does not hold in the current implementation,
 though]
 There are some situations when pages from high memory may need to
 be kmapped, even if bounce buffers are not necessary. For example a device
 may need to abort DMA operations and revert to PIO for the transfer, in
 which case a virtual mapping of the page is required. For SCSI it is also
 done in some scenarios where the low level driver cannot be trusted to
 handle a single sg entry correctly. The driver is expected to perform the
 kmaps as needed on such occasions using the __bio_kmap_atomic and bio_kmap_irq
 routines as appropriate. A driver could also use the blk_queue_bounce()
 routine on its own to bounce highmem i/o to low memory for specific requests
 if so desired.
 iii. The i/o scheduler algorithm itself can be replaced/set as appropriate
 As in 2.4, it is possible to plugin a brand new i/o scheduler for a particular
 queue or pick from (copy) existing generic schedulers and replace/override
 certain portions of it. The 2.5 rewrite provides improved modularization
 of the i/o scheduler. There are more pluggable callbacks, e.g for init,
 add request, extract request, which makes it possible to abstract specific
 i/o scheduling algorithm aspects and details outside of the generic loop.
 It also makes it possible to completely hide the implementation details of
 the i/o scheduler from block drivers.
 I/O scheduler wrappers are to be used instead of accessing the queue directly.
 See section 4. The I/O scheduler for details.
 1.2 Tuning Based on High level code capabilities
 i. Application capabilities for raw i/o
 This comes from some of the high-performance database/middleware
 requirements where an application prefers to make its own i/o scheduling
 decisions based on an understanding of the access patterns and i/o
 characteristics
 ii. High performance filesystems or other higher level kernel code's
 capabilities
 Kernel components like filesystems could also take their own i/o scheduling
 decisions for optimizing performance. Journalling filesystems may need
 some control over i/o ordering.
 What kind of support exists at the generic block layer for this ?
 The flags and rw fields in the bio structure can be used for some tuning
 from above e.g indicating that an i/o is just a readahead request, or for
 marking  barrier requests (discussed next), or priority settings (currently
 unused). As far as user applications are concerned they would need an
 additional mechanism either via open flags or ioctls, or some other upper
 level mechanism to communicate such settings to block.
 1.2.1 I/O Barriers
 There is a way to enforce strict ordering for i/os through barriers.
 All requests before a barrier point must be serviced before the barrier
 request and any other requests arriving after the barrier will not be
 serviced until after the barrier has completed. This is useful for higher
 level control on write ordering, e.g flushing a log of committed updates
 to disk before the corresponding updates themselves.
 A flag in the bio structure, BIO_BARRIER is used to identify a barrier i/o.
 The generic i/o scheduler would make sure that it places the barrier request and
 all other requests coming after it after all the previous requests in the
 queue. Barriers may be implemented in different ways depending on the
 driver. For more details regarding I/O barriers, please read barrier.txt
 in this directory.
 1.2.2 Request Priority/Latency
 Todo/Under discussion:
 Arjan's proposed request priority scheme allows higher levels some broad
  control (high/med/low) over the priority  of an i/o request vs other pending
  requests in the queue. For example it allows reads for bringing in an
  executable page on demand to be given a higher priority over pending write
  requests which haven't aged too much on the queue. Potentially this priority
  could even be exposed to applications in some manner, providing higher level
  tunability. Time based aging avoids starvation of lower priority
  requests. Some bits in the bi_rw flags field in the bio structure are
  intended to be used for this priority information.
 1.3 Direct Access to Low level Device/Driver Capabilities (Bypass mode)
    (e.g Diagnostics, Systems Management)
 There are situations where high-level code needs to have direct access to
 the low level device capabilities or requires the ability to issue commands
 to the device bypassing some of the intermediate i/o layers.
 These could, for example, be special control commands issued through ioctl
 interfaces, or could be raw read/write commands that stress the drive's
 capabilities for certain kinds of fitness tests. Having direct interfaces at
 multiple levels without having to pass through upper layers makes
 it possible to perform bottom up validation of the i/o path, layer by
 layer, starting from the media.
 The normal i/o submission interfaces, e.g submit_bio, could be bypassed
 for specially crafted requests which such ioctl or diagnostics
 interfaces would typically use, and the elevator add_request routine
 can instead be used to directly insert such requests in the queue or preferably
 the blk_do_rq routine can be used to place the request on the queue and
 wait for completion. Alternatively, sometimes the caller might just
 invoke a lower level driver specific interface with the request as a
 parameter.
 If the request is a means for passing on special information associated with
 the command, then such information is associated with the request->special
 field (rather than misuse the request->buffer field which is meant for the
 request data buffer's virtual mapping).
 For passing request data, the caller must build up a bio descriptor
 representing the concerned memory buffer if the underlying driver interprets
 bio segments or uses the block layer end*request* functions for i/o
 completion. Alternatively one could directly use the request->buffer field to
 specify the virtual address of the buffer, if the driver expects buffer
 addresses passed in this way and ignores bio entries for the request type
 involved. In the latter case, the driver would modify and manage the
 request->buffer, request->sector and request->nr_sectors or
 request->current_nr_sectors fields itself rather than using the block layer
 end_request or end_that_request_first completion interfaces.
 (See 2.3 or Documentation/block/request.txt for a brief explanation of
 the request structure fields)
 [TBD: end_that_request_last should be usable even in this case;
 Perhaps an end_that_direct_request_first routine could be implemented to make
 handling direct requests easier for such drivers; Also for drivers that
 expect bios, a helper function could be provided for setting up a bio
 corresponding to a data buffer]
 <JENS: I dont understand the above, why is end_that_request_first() not
 usable? Or _last for that matter. I must be missing something>
 <SUP: What I meant here was that if the request doesn't have a bio, then
 end_that_request_first doesn't modify nr_sectors or current_nr_sectors,
 and hence can't be used for advancing request state settings on the
 completion of partial transfers. The driver has to modify these fields 
 directly by hand.
 This is because end_that_request_first only iterates over the bio list,
 and always returns 0 if there are none associated with the request.
 _last works OK in this case, and is not a problem, as I mentioned earlier
 >
 1.3.1 Pre-built Commands
 A request can be created with a pre-built custom command  to be sent directly
 to the device. The cmd block in the request structure has room for filling
 in the command bytes. (i.e rq->cmd is now 16 bytes in size, and meant for
 command pre-building, and the type of the request is now indicated
 through rq->flags instead of via rq->cmd)
 The request structure flags can be set up to indicate the type of request
 in such cases (REQ_PC: direct packet command passed to driver, REQ_BLOCK_PC:
 packet command issued via blk_do_rq, REQ_SPECIAL: special request).
 It can help to pre-build device commands for requests in advance.
 Drivers can now specify a request prepare function (q->prep_rq_fn) that the
 block layer would invoke to pre-build device commands for a given request,
 or perform other preparatory processing for the request. This is routine is
 called by elv_next_request(), i.e. typically just before servicing a request.
 (The prepare function would not be called for requests that have REQ_DONTPREP
 enabled)
 Aside:
  Pre-building could possibly even be done early, i.e before placing the
  request on the queue, rather than construct the command on the fly in the
  driver while servicing the request queue when it may affect latencies in
  interrupt context or responsiveness in general. One way to add early
  pre-building would be to do it whenever we fail to merge on a request.
  Now REQ_NOMERGE is set in the request flags to skip this one in the future,
  which means that it will not change before we feed it to the device. So
  the pre-builder hook can be invoked there.
 2. Flexible and generic but minimalist i/o structure/descriptor.
 2.1 Reason for a new structure and requirements addressed
 Prior to 2.5, buffer heads were used as the unit of i/o at the generic block
 layer, and the low level request structure was associated with a chain of
 buffer heads for a contiguous i/o request. This led to certain inefficiencies
 when it came to large i/o requests and readv/writev style operations, as it
 forced such requests to be broken up into small chunks before being passed
 on to the generic block layer, only to be merged by the i/o scheduler
 when the underlying device was capable of handling the i/o in one shot.
 Also, using the buffer head as an i/o structure for i/os that didn't originate
 from the buffer cache unecessarily added to the weight of the descriptors
 which were generated for each such chunk.
 The following were some of the goals and expectations considered in the
 redesign of the block i/o data structure in 2.5.
 i.  Should be appropriate as a descriptor for both raw and buffered i/o  -
    avoid cache related fields which are irrelevant in the direct/page i/o path,
    or filesystem block size alignment restrictions which may not be relevant
    for raw i/o.
 ii. Ability to represent high-memory buffers (which do not have a virtual
    address mapping in kernel address space).
 iii.Ability to represent large i/os w/o unecessarily breaking them up (i.e
    greater than PAGE_SIZE chunks in one shot)
 iv. At the same time, ability to retain independent identity of i/os from
    different sources or i/o units requiring individual completion (e.g. for
    latency reasons)
 v.  Ability to represent an i/o involving multiple physical memory segments
    (including non-page aligned page fragments, as specified via readv/writev)
    without unecessarily breaking it up, if the underlying device is capable of
    handling it.
 vi. Preferably should be based on a memory descriptor structure that can be
    passed around different types of subsystems or layers, maybe even
    networking, without duplication or extra copies of data/descriptor fields
    themselves in the process
 vii.Ability to handle the possibility of splits/merges as the structure passes
    through layered drivers (lvm, md, evms), with minimal overhead.
 The solution was to define a new structure (bio)  for the block layer,
 instead of using the buffer head structure (bh) directly, the idea being
 avoidance of some associated baggage and limitations. The bio structure
 is uniformly used for all i/o at the block layer ; it forms a part of the
 bh structure for buffered i/o, and in the case of raw/direct i/o kiobufs are
 mapped to bio structures.
 2.2 The bio struct
 The bio structure uses a vector representation pointing to an array of tuples
 of <page, offset, len> to describe the i/o buffer, and has various other
 fields describing i/o parameters and state that needs to be maintained for
 performing the i/o.
 Notice that this representation means that a bio has no virtual address
 mapping at all (unlike buffer heads).
 struct bio_vec {
       struct page     *bv_page;
       unsigned short  bv_len;
       unsigned short  bv_offset;
 };
 /*
 * main unit of I/O for the block layer and lower layers (ie drivers)
 */
 struct bio {
       sector_t            bi_sector;
       struct bio          *bi_next;    /* request queue link */
       struct block_device *bi_bdev;	/* target device */
       unsigned long       bi_flags;    /* status, command, etc */
       unsigned long       bi_rw;       /* low bits: r/w, high: priority */
       unsigned int	bi_vcnt;     /* how may bio_vec's */
       unsigned int	bi_idx;		/* current index into bio_vec array */
       unsigned int	bi_size;     /* total size in bytes */
       unsigned short 	bi_phys_segments; /* segments after physaddr coalesce*/
       unsigned short	bi_hw_segments; /* segments after DMA remapping */
       unsigned int	bi_max;	     /* max bio_vecs we can hold
                                        used as index into pool */
       struct bio_vec   *bi_io_vec;  /* the actual vec list */
       bio_end_io_t	*bi_end_io;  /* bi_end_io (bio) */
       atomic_t		bi_cnt;	     /* pin count: free when it hits zero */
       void             *bi_private;
       bio_destructor_t *bi_destructor; /* bi_destructor (bio) */
 };
 With this multipage bio design:
 - Large i/os can be sent down in one go using a bio_vec list consisting
  of an array of <page, offset, len> fragments (similar to the way fragments
  are represented in the zero-copy network code)
 - Splitting of an i/o request across multiple devices (as in the case of
  lvm or raid) is achieved by cloning the bio (where the clone points to
  the same bi_io_vec array, but with the index and size accordingly modified)
 - A linked list of bios is used as before for unrelated merges (*) - this
  avoids reallocs and makes independent completions easier to handle.
 - Code that traverses the req list needs to make a distinction between
  segments of a request (bio_for_each_segment) and the distinct completion
  units/bios (rq_for_each_bio).
 - Drivers which can't process a large bio in one shot can use the bi_idx
  field to keep track of the next bio_vec entry to process.
  (e.g a 1MB bio_vec needs to be handled in max 128kB chunks for IDE)
  [TBD: Should preferably also have a bi_voffset and bi_vlen to avoid modifying
   bi_offset an len fields]
 (*) unrelated merges -- a request ends up containing two or more bios that
    didn't originate from the same place.
 bi_end_io() i/o callback gets called on i/o completion of the entire bio.
 At a lower level, drivers build a scatter gather list from the merged bios.
 The scatter gather list is in the form of an array of <page, offset, len>
 entries with their corresponding dma address mappings filled in at the
 appropriate time. As an optimization, contiguous physical pages can be
 covered by a single entry where <page> refers to the first page and <len>
 covers the range of pages (upto 16 contiguous pages could be covered this
 way). There is a helper routine (blk_rq_map_sg) which drivers can use to build
 the sg list.
 Note: Right now the only user of bios with more than one page is ll_rw_kio,
 which in turn means that only raw I/O uses it (direct i/o may not work
 right now). The intent however is to enable clustering of pages etc to
 become possible. The pagebuf abstraction layer from SGI also uses multi-page
 bios, but that is currently not included in the stock development kernels.
 The same is true of Andrew Morton's work-in-progress multipage bio writeout 
 and readahead patches.
 2.3 Changes in the Request Structure
 The request structure is the structure that gets passed down to low level
 drivers. The block layer make_request function builds up a request structure,
 places it on the queue and invokes the drivers request_fn. The driver makes
 use of block layer helper routine elv_next_request to pull the next request
 off the queue. Control or diagnostic functions might bypass block and directly
 invoke underlying driver entry points passing in a specially constructed
 request structure.
 Only some relevant fields (mainly those which changed or may be referred
 to in some of the discussion here) are listed below, not necessarily in
 the order in which they occur in the structure (see include/linux/blkdev.h)
 Refer to Documentation/block/request.txt for details about all the request
 structure fields and a quick reference about the layers which are
 supposed to use or modify those fields.
 struct request {
 	struct list_head queuelist;  /* Not meant to be directly accessed by
 					the driver.
 					Used by q->elv_next_request_fn
 					rq->queue is gone
 					*/
 	.
 	.
 	unsigned char cmd[16]; /* prebuilt command data block */
 	unsigned long flags;   /* also includes earlier rq->cmd settings */
 	.
 	.
 	sector_t sector; /* this field is now of type sector_t instead of int
 			    preparation for 64 bit sectors */
 	.
 	.
 	/* Number of scatter-gather DMA addr+len pairs after
 	 * physical address coalescing is performed.
 	 */
 	unsigned short nr_phys_segments;
 	/* Number of scatter-gather addr+len pairs after
 	 * physical and DMA remapping hardware coalescing is performed.
 	 * This is the number of scatter-gather entries the driver
 	 * will actually have to deal with after DMA mapping is done.
 	 */
 	unsigned short nr_hw_segments;
 	/* Various sector counts */
 	unsigned long nr_sectors;  /* no. of sectors left: driver modifiable */
 	unsigned long hard_nr_sectors;  /* block internal copy of above */
 	unsigned int current_nr_sectors; /* no. of sectors left in the
 					   current segment:driver modifiable */
 	unsigned long hard_cur_sectors; /* block internal copy of the above */
 	.
 	.
 	int tag;	/* command tag associated with request */
 	void *special;  /* same as before */
 	char *buffer;   /* valid only for low memory buffers upto
 			 current_nr_sectors */
 	.
 	.
 	struct bio *bio, *biotail;  /* bio list instead of bh */
 	struct request_list *rl;
 }
 See the rq_flag_bits definitions for an explanation of the various flags
 available. Some bits are used by the block layer or i/o scheduler.
 The behaviour of the various sector counts are almost the same as before,
 except that since we have multi-segment bios, current_nr_sectors refers
 to the numbers of sectors in the current segment being processed which could
 be one of the many segments in the current bio (i.e i/o completion unit).
 The nr_sectors value refers to the total number of sectors in the whole
 request that remain to be transferred (no change). The purpose of the
 hard_xxx values is for block to remember these counts every time it hands
 over the request to the driver. These values are updated by block on
 end_that_request_first, i.e. every time the driver completes a part of the
 transfer and invokes block end*request helpers to mark this. The
 driver should not modify these values. The block layer sets up the
 nr_sectors and current_nr_sectors fields (based on the corresponding
 hard_xxx values and the number of bytes transferred) and updates it on
 every transfer that invokes end_that_request_first. It does the same for the
 buffer, bio, bio->bi_idx fields too.
 The buffer field is just a virtual address mapping of the current segment
 of the i/o buffer in cases where the buffer resides in low-memory. For high
 memory i/o, this field is not valid and must not be used by drivers.
 Code that sets up its own request structures and passes them down to
 a driver needs to be careful about interoperation with the block layer helper
 functions which the driver uses. (Section 1.3)
 3. Using bios
 3.1 Setup/Teardown
 There are routines for managing the allocation, and reference counting, and
 freeing of bios (bio_alloc, bio_get, bio_put).
 This makes use of Ingo Molnar's mempool implementation, which enables
 subsystems like bio to maintain their own reserve memory pools for guaranteed
 deadlock-free allocations during extreme VM load. For example, the VM
 subsystem makes use of the block layer to writeout dirty pages in order to be
 able to free up memory space, a case which needs careful handling. The
 allocation logic draws from the preallocated emergency reserve in situations
 where it cannot allocate through normal means. If the pool is empty and it
 can wait, then it would trigger action that would help free up memory or
 replenish the pool (without deadlocking) and wait for availability in the pool.
 If it is in IRQ context, and hence not in a position to do this, allocation
 could fail if the pool is empty. In general mempool always first tries to
 perform allocation without having to wait, even if it means digging into the
 pool as long it is not less that 50% full.
 On a free, memory is released to the pool or directly freed depending on
 the current availability in the pool. The mempool interface lets the
 subsystem specify the routines to be used for normal alloc and free. In the
 case of bio, these routines make use of the standard slab allocator.
 The caller of bio_alloc is expected to taken certain steps to avoid
 deadlocks, e.g. avoid trying to allocate more memory from the pool while
 already holding memory obtained from the pool.
 [TBD: This is a potential issue, though a rare possibility
 in the bounce bio allocation that happens in the current code, since
 it ends up allocating a second bio from the same pool while
 holding the original bio ]
 Memory allocated from the pool should be released back within a limited
 amount of time (in the case of bio, that would be after the i/o is completed).
 This ensures that if part of the pool has been used up, some work (in this
 case i/o) must already be in progress and memory would be available when it
 is over. If allocating from multiple pools in the same code path, the order
 or hierarchy of allocation needs to be consistent, just the way one deals
 with multiple locks.
 The bio_alloc routine also needs to allocate the bio_vec_list (bvec_alloc())
 for a non-clone bio. There are the 6 pools setup for different size biovecs,
 so bio_alloc(gfp_mask, nr_iovecs) will allocate a vec_list of the
 given size from these slabs.
 The bi_destructor() routine takes into account the possibility of the bio
 having originated from a different source (see later discussions on
 n/w to block transfers and kvec_cb)
 The bio_get() routine may be used to hold an extra reference on a bio prior
 to i/o submission, if the bio fields are likely to be accessed after the
 i/o is issued (since the bio may otherwise get freed in case i/o completion
 happens in the meantime).
 The bio_clone() routine may be used to duplicate a bio, where the clone
 shares the bio_vec_list with the original bio (i.e. both point to the
 same bio_vec_list). This would typically be used for splitting i/o requests
 in lvm or md.
 3.2 Generic bio helper Routines
 3.2.1 Traversing segments and completion units in a request
 The macros bio_for_each_segment() and rq_for_each_bio() should be used for
 traversing the bios in the request list (drivers should avoid directly
 trying to do it themselves). Using these helpers should also make it easier
 to cope with block changes in the future.
 	rq_for_each_bio(bio, rq)
 		bio_for_each_segment(bio_vec, bio, i)
 			/* bio_vec is now current segment */
 I/O completion callbacks are per-bio rather than per-segment, so drivers
 that traverse bio chains on completion need to keep that in mind. Drivers
 which don't make a distinction between segments and completion units would
 need to be reorganized to support multi-segment bios.
 3.2.2 Setting up DMA scatterlists
 The blk_rq_map_sg() helper routine would be used for setting up scatter
 gather lists from a request, so a driver need not do it on its own.
 	nr_segments = blk_rq_map_sg(q, rq, scatterlist);
 The helper routine provides a level of abstraction which makes it easier
 to modify the internals of request to scatterlist conversion down the line
 without breaking drivers. The blk_rq_map_sg routine takes care of several
 things like collapsing physically contiguous segments (if QUEUE_FLAG_CLUSTER
 is set) and correct segment accounting to avoid exceeding the limits which
 the i/o hardware can handle, based on various queue properties.
 - Prevents a clustered segment from crossing a 4GB mem boundary
 - Avoids building segments that would exceed the number of physical
  memory segments that the driver can handle (phys_segments) and the
  number that the underlying hardware can handle at once, accounting for
  DMA remapping (hw_segments)  (i.e. IOMMU aware limits).
 Routines which the low level driver can use to set up the segment limits:
 blk_queue_max_hw_segments() : Sets an upper limit of the maximum number of
 hw data segments in a request (i.e. the maximum number of address/length
 pairs the host adapter can actually hand to the device at once)
 blk_queue_max_phys_segments() : Sets an upper limit on the maximum number
 of physical data segments in a request (i.e. the largest sized scatter list
 a driver could handle)
 3.2.3 I/O completion
 The existing generic block layer helper routines end_request,
 end_that_request_first and end_that_request_last can be used for i/o
 completion (and setting things up so the rest of the i/o or the next
 request can be kicked of) as before. With the introduction of multi-page
 bio support, end_that_request_first requires an additional argument indicating
 the number of sectors completed.
 3.2.4 Implications for drivers that do not interpret bios (don't handle
 multiple segments)
 Drivers that do not interpret bios e.g those which do not handle multiple
 segments and do not support i/o into high memory addresses (require bounce
 buffers) and expect only virtually mapped buffers, can access the rq->buffer
 field. As before the driver should use current_nr_sectors to determine the
 size of remaining data in the current segment (that is the maximum it can
 transfer in one go unless it interprets segments), and rely on the block layer
 end_request, or end_that_request_first/last to take care of all accounting
 and transparent mapping of the next bio segment when a segment boundary
 is crossed on completion of a transfer. (The end*request* functions should
 be used if only if the request has come down from block/bio path, not for
 direct access requests which only specify rq->buffer without a valid rq->bio)
 3.2.5 Generic request command tagging
 3.2.5.1 Tag helpers
 Block now offers some simple generic functionality to help support command
 queueing (typically known as tagged command queueing), ie manage more than
 one outstanding command on a queue at any given time.
 	blk_queue_init_tags(request_queue_t *q, int depth)
 	Initialize internal command tagging structures for a maximum
 	depth of 'depth'.
 	blk_queue_free_tags((request_queue_t *q)
 	Teardown tag info associated with the queue. This will be done
 	automatically by block if blk_queue_cleanup() is called on a queue
 	that is using tagging.
 The above are initialization and exit management, the main helpers during
 normal operations are:
 	blk_queue_start_tag(request_queue_t *q, struct request *rq)
 	Start tagged operation for this request. A free tag number between
 	0 and 'depth' is assigned to the request (rq->tag holds this number),
 	and 'rq' is added to the internal tag management. If the maximum depth
 	for this queue is already achieved (or if the tag wasn't started for
 	some other reason), 1 is returned. Otherwise 0 is returned.
 	blk_queue_end_tag(request_queue_t *q, struct request *rq)
 	End tagged operation on this request. 'rq' is removed from the internal
 	book keeping structures.
 To minimize struct request and queue overhead, the tag helpers utilize some
 of the same request members that are used for normal request queue management.
 This means that a request cannot both be an active tag and be on the queue
 list at the same time. blk_queue_start_tag() will remove the request, but
 the driver must remember to call blk_queue_end_tag() before signalling
 completion of the request to the block layer. This means ending tag
 operations before calling end_that_request_last()! For an example of a user
 of these helpers, see the IDE tagged command queueing support.
 Certain hardware conditions may dictate a need to invalidate the block tag
 queue. For instance, on IDE any tagged request error needs to clear both
 the hardware and software block queue and enable the driver to sanely restart
 all the outstanding requests. There's a third helper to do that:
 	blk_queue_invalidate_tags(request_queue_t *q)
 	Clear the internal block tag queue and re-add all the pending requests
 	to the request queue. The driver will receive them again on the
 	next request_fn run, just like it did the first time it encountered
 	them.
 3.2.5.2 Tag info
 Some block functions exist to query current tag status or to go from a
 tag number to the associated request. These are, in no particular order:
 	blk_queue_tagged(q)
 	Returns 1 if the queue 'q' is using tagging, 0 if not.
 	blk_queue_tag_request(q, tag)
 	Returns a pointer to the request associated with tag 'tag'.
 	blk_queue_tag_depth(q)
 	Return current queue depth.
 	blk_queue_tag_queue(q)
 	Returns 1 if the queue can accept a new queued command, 0 if we are
 	at the maximum depth already.
 	blk_queue_rq_tagged(rq)
 	Returns 1 if the request 'rq' is tagged.
 3.2.5.2 Internal structure
 Internally, block manages tags in the blk_queue_tag structure:
 	struct blk_queue_tag {
 		struct request **tag_index;	/* array or pointers to rq */
 		unsigned long *tag_map;		/* bitmap of free tags */
 		struct list_head busy_list;	/* fifo list of busy tags */
 		int busy;			/* queue depth */
 		int max_depth;			/* max queue depth */
 	};
 Most of the above is simple and straight forward, however busy_list may need
 a bit of explaining. Normally we don't care too much about request ordering,
 but in the event of any barrier requests in the tag queue we need to ensure
 that requests are restarted in the order they were queue. This may happen
 if the driver needs to use blk_queue_invalidate_tags().
 Tagging also defines a new request flag, REQ_QUEUED. This is set whenever
 a request is currently tagged. You should not use this flag directly,
 blk_rq_tagged(rq) is the portable way to do so.
 3.3 I/O Submission
 The routine submit_bio() is used to submit a single io. Higher level i/o
 routines make use of this:
 (a) Buffered i/o:
 The routine submit_bh() invokes submit_bio() on a bio corresponding to the
 bh, allocating the bio if required. ll_rw_block() uses submit_bh() as before.
 (b) Kiobuf i/o (for raw/direct i/o):
 The ll_rw_kio() routine breaks up the kiobuf into page sized chunks and
 maps the array to one or more multi-page bios, issuing submit_bio() to
 perform the i/o on each of these.
 The embedded bh array in the kiobuf structure has been removed and no
 preallocation of bios is done for kiobufs. [The intent is to remove the
 blocks array as well, but it's currently in there to kludge around direct i/o.]
 Thus kiobuf allocation has switched back to using kmalloc rather than vmalloc.
 Todo/Observation:
 A single kiobuf structure is assumed to correspond to a contiguous range
 of data, so brw_kiovec() invokes ll_rw_kio for each kiobuf in a kiovec.
 So right now it wouldn't work for direct i/o on non-contiguous blocks.
 This is to be resolved.  The eventual direction is to replace kiobuf
 by kvec's.
 Badari Pulavarty has a patch to implement direct i/o correctly using
 bio and kvec.
 (c) Page i/o:
 Todo/Under discussion:
 Andrew Morton's multi-page bio patches attempt to issue multi-page
 writeouts (and reads) from the page cache, by directly building up
 large bios for submission completely bypassing the usage of buffer
 heads. This work is still in progress.
 Christoph Hellwig had some code that uses bios for page-io (rather than
 bh). This isn't included in bio as yet. Christoph was also working on a
 design for representing virtual/real extents as an entity and modifying
 some of the address space ops interfaces to utilize this abstraction rather
 than buffer_heads. (This is somewhat along the lines of the SGI XFS pagebuf
 abstraction, but intended to be as lightweight as possible).
 (d) Direct access i/o:
 Direct access requests that do not contain bios would be submitted differently
 as discussed earlier in section 1.3.
 Aside:
  Kvec i/o:
  Ben LaHaise's aio code uses a slightly different structure instead
  of kiobufs, called a kvec_cb. This contains an array of <page, offset, len>
  tuples (very much like the networking code), together with a callback function
  and data pointer. This is embedded into a brw_cb structure when passed
  to brw_kvec_async().
  Now it should be possible to directly map these kvecs to a bio. Just as while
  cloning, in this case rather than PRE_BUILT bio_vecs, we set the bi_io_vec
  array pointer to point to the veclet array in kvecs.
  TBD: In order for this to work, some changes are needed in the way multi-page
  bios are handled today. The values of the tuples in such a vector passed in
  from higher level code should not be modified by the block layer in the course
  of its request processing, since that would make it hard for the higher layer
  to continue to use the vector descriptor (kvec) after i/o completes. Instead,
  all such transient state should either be maintained in the request structure,
  and passed on in some way to the endio completion routine.
 4. The I/O scheduler
 I/O scheduler, a.k.a. elevator, is implemented in two layers.  Generic dispatch
 queue and specific I/O schedulers.  Unless stated otherwise, elevator is used
 to refer to both parts and I/O scheduler to specific I/O schedulers.
 Block layer implements generic dispatch queue in ll_rw_blk.c and elevator.c.
 The generic dispatch queue is responsible for properly ordering barrier
 requests, requeueing, handling non-fs requests and all other subtleties.
 Specific I/O schedulers are responsible for ordering normal filesystem
 requests.  They can also choose to delay certain requests to improve
 throughput or whatever purpose.  As the plural form indicates, there are
 multiple I/O schedulers.  They can be built as modules but at least one should
 be built inside the kernel.  Each queue can choose different one and can also
 change to another one dynamically.
 A block layer call to the i/o scheduler follows the convention elv_xxx(). This
 calls elevator_xxx_fn in the elevator switch (drivers/block/elevator.c). Oh,
 xxx and xxx might not match exactly, but use your imagination. If an elevator
 doesn't implement a function, the switch does nothing or some minimal house
 keeping work.
 4.1. I/O scheduler API
 The functions an elevator may implement are: (* are mandatory)
 elevator_merge_fn		called to query requests for merge with a bio
 elevator_merge_req_fn		called when two requests get merged. the one
 				which gets merged into the other one will be
 				never seen by I/O scheduler again. IOW, after
 				being merged, the request is gone.
 elevator_merged_fn		called when a request in the scheduler has been
 				involved in a merge. It is used in the deadline
 				scheduler for example, to reposition the request
 				if its sorting order has changed.
 elevator_dispatch_fn		fills the dispatch queue with ready requests.
 				I/O schedulers are free to postpone requests by
 				not filling the dispatch queue unless @force
 				is non-zero.  Once dispatched, I/O schedulers
 				are not allowed to manipulate the requests -
 				they belong to generic dispatch queue.
 elevator_add_req_fn		called to add a new request into the scheduler
 elevator_queue_empty_fn		returns true if the merge queue is empty.
 				Drivers shouldn't use this, but rather check
 				if elv_next_request is NULL (without losing the
 				request if one exists!)
 elevator_former_req_fn
 elevator_latter_req_fn		These return the request before or after the
 				one specified in disk sort order. Used by the
 				block layer to find merge possibilities.
 elevator_completed_req_fn	called when a request is completed.
 elevator_may_queue_fn		returns true if the scheduler wants to allow the
 				current context to queue a new request even if
 				it is over the queue limit. This must be used
 				very carefully!!
 elevator_set_req_fn
 elevator_put_req_fn		Must be used to allocate and free any elevator
 				specific storage for a request.
 elevator_activate_req_fn	Called when device driver first sees a request.
 				I/O schedulers can use this callback to
 				determine when actual execution of a request
 				starts.
 elevator_deactivate_req_fn	Called when device driver decides to delay
 				a request by requeueing it.
 elevator_init_fn
 elevator_exit_fn		Allocate and free any elevator specific storage
 				for a queue.
 4.2 Request flows seen by I/O schedulers
 All requests seen by I/O schedulers strictly follow one of the following three
 flows.
 set_req_fn ->
 i.   add_req_fn -> (merged_fn ->)* -> dispatch_fn -> activate_req_fn ->
      (deactivate_req_fn -> activate_req_fn ->)* -> completed_req_fn
 ii.  add_req_fn -> (merged_fn ->)* -> merge_req_fn
 iii. [none]
 -> put_req_fn
 4.3 I/O scheduler implementation
 The generic i/o scheduler algorithm attempts to sort/merge/batch requests for
 optimal disk scan and request servicing performance (based on generic
 principles and device capabilities), optimized for:
 i.   improved throughput
 ii.  improved latency
 iii. better utilization of h/w & CPU time
 Characteristics:
 i. Binary tree
 AS and deadline i/o schedulers use red black binary trees for disk position
 sorting and searching, and a fifo linked list for time-based searching. This
 gives good scalability and good availablility of information. Requests are
 almost always dispatched in disk sort order, so a cache is kept of the next
 request in sort order to prevent binary tree lookups.
 This arrangement is not a generic block layer characteristic however, so
 elevators may implement queues as they please.
 ii. Merge hash
 AS and deadline use a hash table indexed by the last sector of a request. This
 enables merging code to quickly look up "back merge" candidates, even when
 multiple I/O streams are being performed at once on one disk.
 "Front merges", a new request being merged at the front of an existing request,
 are far less common than "back merges" due to the nature of most I/O patterns.
 Front merges are handled by the binary trees in AS and deadline schedulers.
 iii. Plugging the queue to batch requests in anticipation of opportunities for
     merge/sort optimizations
 This is just the same as in 2.4 so far, though per-device unplugging
 support is anticipated for 2.5. Also with a priority-based i/o scheduler,
 such decisions could be based on request priorities.
 Plugging is an approach that the current i/o scheduling algorithm resorts to so
 that it collects up enough requests in the queue to be able to take
 advantage of the sorting/merging logic in the elevator. If the
 queue is empty when a request comes in, then it plugs the request queue
 (sort of like plugging the bottom of a vessel to get fluid to build up)
 till it fills up with a few more requests, before starting to service
 the requests. This provides an opportunity to merge/sort the requests before
 passing them down to the device. There are various conditions when the queue is
 unplugged (to open up the flow again), either through a scheduled task or
 could be on demand. For example wait_on_buffer sets the unplugging going
 (by running tq_disk) so the read gets satisfied soon. So in the read case,
 the queue gets explicitly unplugged as part of waiting for completion,
 in fact all queues get unplugged as a side-effect.
 Aside:
  This is kind of controversial territory, as it's not clear if plugging is
  always the right thing to do. Devices typically have their own queues,
  and allowing a big queue to build up in software, while letting the device be
  idle for a while may not always make sense. The trick is to handle the fine
  balance between when to plug and when to open up. Also now that we have
  multi-page bios being queued in one shot, we may not need to wait to merge
  a big request from the broken up pieces coming by.
  Per-queue granularity unplugging (still a Todo) may help reduce some of the
  concerns with just a single tq_disk flush approach. Something like
  blk_kick_queue() to unplug a specific queue (right away ?)
  or optionally, all queues, is in the plan.
 4.4 I/O contexts
 I/O contexts provide a dynamically allocated per process data area. They may
 be used in I/O schedulers, and in the block layer (could be used for IO statis,
 priorities for example). See *io_context in block/ll_rw_blk.c, and as-iosched.c
 for an example of usage in an i/o scheduler.
 5. Scalability related changes
 5.1 Granular Locking: io_request_lock replaced by a per-queue lock
 The global io_request_lock has been removed as of 2.5, to avoid
 the scalability bottleneck it was causing, and has been replaced by more
 granular locking. The request queue structure has a pointer to the
 lock to be used for that queue. As a result, locking can now be
 per-queue, with a provision for sharing a lock across queues if
 necessary (e.g the scsi layer sets the queue lock pointers to the
 corresponding adapter lock, which results in a per host locking
 granularity). The locking semantics are the same, i.e. locking is
 still imposed by the block layer, grabbing the lock before
 request_fn execution which it means that lots of older drivers
 should still be SMP safe. Drivers are free to drop the queue
 lock themselves, if required. Drivers that explicitly used the
 io_request_lock for serialization need to be modified accordingly.
 Usually it's as easy as adding a global lock:
 	static spinlock_t my_driver_lock = SPIN_LOCK_UNLOCKED;
 and passing the address to that lock to blk_init_queue().
 5.2 64 bit sector numbers (sector_t prepares for 64 bit support)
 The sector number used in the bio structure has been changed to sector_t,
 which could be defined as 64 bit in preparation for 64 bit sector support.
 6. Other Changes/Implications
 6.1 Partition re-mapping handled by the generic block layer
 In 2.5 some of the gendisk/partition related code has been reorganized.
 Now the generic block layer performs partition-remapping early and thus
 provides drivers with a sector number relative to whole device, rather than
 having to take partition number into account in order to arrive at the true
 sector number. The routine blk_partition_remap() is invoked by
 generic_make_request even before invoking the queue specific make_request_fn,
 so the i/o scheduler also gets to operate on whole disk sector numbers. This
 should typically not require changes to block drivers, it just never gets
 to invoke its own partition sector offset calculations since all bios
 sent are offset from the beginning of the device.
 7. A Few Tips on Migration of older drivers
 Old-style drivers that just use CURRENT and ignores clustered requests,
 may not need much change.  The generic layer will automatically handle
 clustered requests, multi-page bios, etc for the driver.
 For a low performance driver or hardware that is PIO driven or just doesn't
 support scatter-gather changes should be minimal too.
 The following are some points to keep in mind when converting old drivers
 to bio.
 Drivers should use elv_next_request to pick up requests and are no longer
 supposed to handle looping directly over the request list.
 (struct request->queue has been removed)
 Now end_that_request_first takes an additional number_of_sectors argument.
 It used to handle always just the first buffer_head in a request, now
 it will loop and handle as many sectors (on a bio-segment granularity)
 as specified.
 Now bh->b_end_io is replaced by bio->bi_end_io, but most of the time the
 right thing to use is bio_endio(bio, uptodate) instead.
 If the driver is dropping the io_request_lock from its request_fn strategy,
 then it just needs to replace that with q->queue_lock instead.
 As described in Sec 1.1, drivers can set max sector size, max segment size
 etc per queue now. Drivers that used to define their own merge functions i
 to handle things like this can now just use the blk_queue_* functions at
 blk_init_queue time.
 Drivers no longer have to map a {partition, sector offset} into the
 correct absolute location anymore, this is done by the block layer, so
 where a driver received a request ala this before:
 	rq->rq_dev = mk_kdev(3, 5);	/* /dev/hda5 */
 	rq->sector = 0;			/* first sector on hda5 */
  it will now see
 	rq->rq_dev = mk_kdev(3, 0);	/* /dev/hda */
 	rq->sector = 123128;		/* offset from start of disk */
 As mentioned, there is no virtual mapping of a bio. For DMA, this is
 not a problem as the driver probably never will need a virtual mapping.
 Instead it needs a bus mapping (pci_map_page for a single segment or
 use blk_rq_map_sg for scatter gather) to be able to ship it to the driver. For
 PIO drivers (or drivers that need to revert to PIO transfer once in a
 while (IDE for example)), where the CPU is doing the actual data
 transfer a virtual mapping is needed. If the driver supports highmem I/O,
 (Sec 1.1, (ii) ) it needs to use __bio_kmap_atomic and bio_kmap_irq to
 temporarily map a bio into the virtual address space.
 8. Prior/Related/Impacted patches
 8.1. Earlier kiobuf patches (sct/axboe/chait/hch/mkp)
 - orig kiobuf & raw i/o patches (now in 2.4 tree)
 - direct kiobuf based i/o to devices (no intermediate bh's)
 - page i/o using kiobuf
 - kiobuf splitting for lvm (mkp)
 - elevator support for kiobuf request merging (axboe)
 8.2. Zero-copy networking (Dave Miller)
 8.3. SGI XFS - pagebuf patches - use of kiobufs
 8.4. Multi-page pioent patch for bio (Christoph Hellwig)
 8.5. Direct i/o implementation (Andrea Arcangeli) since 2.4.10-pre11
 8.6. Async i/o implementation patch (Ben LaHaise)
 8.7. EVMS layering design (IBM EVMS team)
 8.8. Larger page cache size patch (Ben LaHaise) and
     Large page size (Daniel Phillips)
    => larger contiguous physical memory buffers
 8.9. VM reservations patch (Ben LaHaise)
 8.10. Write clustering patches ? (Marcelo/Quintela/Riel ?)
 8.11. Block device in page cache patch (Andrea Archangeli) - now in 2.4.10+
 8.12. Multiple block-size transfers for faster raw i/o (Shailabh Nagar,
      Badari)
 8.13  Priority based i/o scheduler - prepatches (Arjan van de Ven)
 8.14  IDE Taskfile i/o patch (Andre Hedrick)
 8.15  Multi-page writeout and readahead patches (Andrew Morton)
 8.16  Direct i/o patches for 2.5 using kvec and bio (Badari Pulavarthy)
 9. Other References:
 9.1 The Splice I/O Model - Larry McVoy (and subsequent discussions on lkml,
 and Linus' comments - Jan 2001)
 9.2 Discussions about kiobuf and bh design on lkml between sct, linus, alan
 et al - Feb-March 2001 (many of the initial thoughts that led to bio were
 brought up in this discussion thread)
 9.3 Discussions on mempool on lkml - Dec 2001.
 The Linux Kernel Device Model
 Patrick Mochel	<mochel@digitalimplant.org>
 Drafted 26 August 2002
 Updated 31 January 2006
 Overview
 ~~~~~~~~
 The Linux Kernel Driver Model is a unification of all the disparate driver
 models that were previously used in the kernel. It is intended to augment the
 bus-specific drivers for bridges and devices by consolidating a set of data
 and operations into globally accessible data structures.
 Traditional driver models implemented some sort of tree-like structure
 (sometimes just a list) for the devices they control. There wasn't any
 uniformity across the different bus types.
 The current driver model provides a common, uniform data model for describing
 a bus and the devices that can appear under the bus. The unified bus
 model includes a set of common attributes which all busses carry, and a set
 of common callbacks, such as device discovery during bus probing, bus
 shutdown, bus power management, etc.
 The common device and bridge interface reflects the goals of the modern
 computer: namely the ability to do seamless device "plug and play", power
 management, and hot plug. In particular, the model dictated by Intel and
 Microsoft (namely ACPI) ensures that almost every device on almost any bus
 on an x86-compatible system can work within this paradigm.  Of course,
 not every bus is able to support all such operations, although most
 buses support a most of those operations.
 Downstream Access
 ~~~~~~~~~~~~~~~~~
 Common data fields have been moved out of individual bus layers into a common
 data structure. These fields must still be accessed by the bus layers,
 and sometimes by the device-specific drivers.
 Other bus layers are encouraged to do what has been done for the PCI layer.
 struct pci_dev now looks like this:
 struct pci_dev {
 	...
 	struct device dev;
 };
 Note first that it is statically allocated. This means only one allocation on
 device discovery. Note also that it is at the _end_ of struct pci_dev. This is
 to make people think about what they're doing when switching between the bus
 driver and the global driver; and to prevent against mindless casts between
 the two.
 The PCI bus layer freely accesses the fields of struct device. It knows about
 the structure of struct pci_dev, and it should know the structure of struct
-device. Individual PCI device drivers that have been converted the the current
+device. Individual PCI device drivers that have been converted to the current
 driver model generally do not and should not touch the fields of struct device,
 unless there is a strong compelling reason to do so.
 This abstraction is prevention of unnecessary pain during transitional phases.
 If the name of the field changes or is removed, then every downstream driver
 will break. On the other hand, if only the bus layer (and not the device
 layer) accesses struct device, it is only that layer that needs to change.
 User Interface
 ~~~~~~~~~~~~~~
 By virtue of having a complete hierarchical view of all the devices in the
 system, exporting a complete hierarchical view to userspace becomes relatively
 easy. This has been accomplished by implementing a special purpose virtual
 file system named sysfs. It is hence possible for the user to mount the
 whole sysfs filesystem anywhere in userspace.
 This can be done permanently by providing the following entry into the
 /etc/fstab (under the provision that the mount point does exist, of course):
 none     	/sys	sysfs    defaults		0	0
 Or by hand on the command line:
 # mount -t sysfs sysfs /sys
 Whenever a device is inserted into the tree, a directory is created for it.
 This directory may be populated at each layer of discovery - the global layer,
 the bus layer, or the device layer.
 The global layer currently creates two files - 'name' and 'power'. The
 former only reports the name of the device. The latter reports the
 current power state of the device. It will also be used to set the current
 power state. 
 The bus layer may also create files for the devices it finds while probing the
 bus. For example, the PCI layer currently creates 'irq' and 'resource' files
 for each PCI device.
 A device-specific driver may also export files in its directory to expose
 device-specific data or tunable interfaces.
 More information about the sysfs directory layout can be found in
 the other documents in this directory and in the file 
 Documentation/filesystems/sysfs.txt.
     Kernel level exception handling in Linux 2.1.8
  Commentary by Joerg Pommnitz <joerg@raleigh.ibm.com>
 When a process runs in kernel mode, it often has to access user 
 mode memory whose address has been passed by an untrusted program. 
 To protect itself the kernel has to verify this address.
 In older versions of Linux this was done with the 
 int verify_area(int type, const void * addr, unsigned long size) 
 function (which has since been replaced by access_ok()).
 This function verified that the memory area starting at address 
-addr and of size size was accessible for the operation specified 
+'addr' and of size 'size' was accessible for the operation specified
 in type (read or write). To do this, verify_read had to look up the 
 virtual memory area (vma) that contained the address addr. In the 
 normal case (correctly working program), this test was successful. 
 It only failed for a few buggy programs. In some kernel profiling
 tests, this normally unneeded verification used up a considerable
 amount of time.
 To overcome this situation, Linus decided to let the virtual memory 
 hardware present in every Linux-capable CPU handle this test.
 How does this work?
 Whenever the kernel tries to access an address that is currently not 
 accessible, the CPU generates a page fault exception and calls the 
 page fault handler 
 void do_page_fault(struct pt_regs *regs, unsigned long error_code)
 in arch/i386/mm/fault.c. The parameters on the stack are set up by 
 the low level assembly glue in arch/i386/kernel/entry.S. The parameter
 regs is a pointer to the saved registers on the stack, error_code 
 contains a reason code for the exception.
 do_page_fault first obtains the unaccessible address from the CPU 
 control register CR2. If the address is within the virtual address 
 space of the process, the fault probably occurred, because the page 
 was not swapped in, write protected or something similar. However, 
 we are interested in the other case: the address is not valid, there 
 is no vma that contains this address. In this case, the kernel jumps 
 to the bad_area label. 
 There it uses the address of the instruction that caused the exception 
 (i.e. regs->eip) to find an address where the execution can continue 
 (fixup). If this search is successful, the fault handler modifies the 
 return address (again regs->eip) and returns. The execution will 
 continue at the address in fixup.
 Where does fixup point to?
 Since we jump to the contents of fixup, fixup obviously points 
 to executable code. This code is hidden inside the user access macros. 
 I have picked the get_user macro defined in include/asm/uaccess.h as an
 example. The definition is somewhat hard to follow, so let's peek at 
 the code generated by the preprocessor and the compiler. I selected
 the get_user call in drivers/char/console.c for a detailed examination.
 The original code in console.c line 1405:
        get_user(c, buf);
 The preprocessor output (edited to become somewhat readable):
 (
  {        
    long __gu_err = - 14 , __gu_val = 0;        
    const __typeof__(*( (  buf ) )) *__gu_addr = ((buf));        
    if (((((0 + current_set[0])->tss.segment) == 0x18 )  || 
       (((sizeof(*(buf))) <= 0xC0000000UL) && 
       ((unsigned long)(__gu_addr ) <= 0xC0000000UL - (sizeof(*(buf)))))))        
      do {
        __gu_err  = 0;        
        switch ((sizeof(*(buf)))) {        
          case 1: 
            __asm__ __volatile__(        
              "1:      mov" "b" " %2,%" "b" "1\n"        
              "2:\n"        
              ".section .fixup,\"ax\"\n"        
              "3:      movl %3,%0\n"        
              "        xor" "b" " %" "b" "1,%" "b" "1\n"        
              "        jmp 2b\n"        
              ".section __ex_table,\"a\"\n"        
              "        .align 4\n"        
              "        .long 1b,3b\n"        
              ".text"        : "=r"(__gu_err), "=q" (__gu_val): "m"((*(struct __large_struct *)
                            (   __gu_addr   )) ), "i"(- 14 ), "0"(  __gu_err  )) ; 
              break;        
          case 2: 
            __asm__ __volatile__(
              "1:      mov" "w" " %2,%" "w" "1\n"        
              "2:\n"        
              ".section .fixup,\"ax\"\n"        
              "3:      movl %3,%0\n"        
              "        xor" "w" " %" "w" "1,%" "w" "1\n"        
              "        jmp 2b\n"        
              ".section __ex_table,\"a\"\n"        
              "        .align 4\n"        
              "        .long 1b,3b\n"        
              ".text"        : "=r"(__gu_err), "=r" (__gu_val) : "m"((*(struct __large_struct *)
                            (   __gu_addr   )) ), "i"(- 14 ), "0"(  __gu_err  )); 
              break;        
          case 4: 
            __asm__ __volatile__(        
              "1:      mov" "l" " %2,%" "" "1\n"        
              "2:\n"        
              ".section .fixup,\"ax\"\n"        
              "3:      movl %3,%0\n"        
              "        xor" "l" " %" "" "1,%" "" "1\n"        
              "        jmp 2b\n"        
              ".section __ex_table,\"a\"\n"        
              "        .align 4\n"        "        .long 1b,3b\n"        
              ".text"        : "=r"(__gu_err), "=r" (__gu_val) : "m"((*(struct __large_struct *)
                            (   __gu_addr   )) ), "i"(- 14 ), "0"(__gu_err)); 
              break;        
          default: 
            (__gu_val) = __get_user_bad();        
        }        
      } while (0) ;        
    ((c)) = (__typeof__(*((buf))))__gu_val;        
    __gu_err;
  }
 );
 WOW! Black GCC/assembly magic. This is impossible to follow, so let's
 see what code gcc generates:
 >         xorl %edx,%edx
 >         movl current_set,%eax
 >         cmpl $24,788(%eax)        
 >         je .L1424        
 >         cmpl $-1073741825,64(%esp)
 >         ja .L1423                
 > .L1424:
 >         movl %edx,%eax                        
 >         movl 64(%esp),%ebx
 > #APP
 > 1:      movb (%ebx),%dl                /* this is the actual user access */
 > 2:
 > .section .fixup,"ax"
 > 3:      movl $-14,%eax
 >         xorb %dl,%dl
 >         jmp 2b
 > .section __ex_table,"a"
 >         .align 4
 >         .long 1b,3b
 > .text
 > #NO_APP
 > .L1423:
 >         movzbl %dl,%esi
 The optimizer does a good job and gives us something we can actually 
 understand. Can we? The actual user access is quite obvious. Thanks 
 to the unified address space we can just access the address in user 
 memory. But what does the .section stuff do?????
 To understand this we have to look at the final kernel:
 > objdump --section-headers vmlinux
 > 
 > vmlinux:     file format elf32-i386
 > 
 > Sections:
 > Idx Name          Size      VMA       LMA       File off  Algn
 >   0 .text         00098f40  c0100000  c0100000  00001000  2**4
 >                   CONTENTS, ALLOC, LOAD, READONLY, CODE
 >   1 .fixup        000016bc  c0198f40  c0198f40  00099f40  2**0
 >                   CONTENTS, ALLOC, LOAD, READONLY, CODE
 >   2 .rodata       0000f127  c019a5fc  c019a5fc  0009b5fc  2**2
 >                   CONTENTS, ALLOC, LOAD, READONLY, DATA
 >   3 __ex_table    000015c0  c01a9724  c01a9724  000aa724  2**2
 >                   CONTENTS, ALLOC, LOAD, READONLY, DATA
 >   4 .data         0000ea58  c01abcf0  c01abcf0  000abcf0  2**4
 >                   CONTENTS, ALLOC, LOAD, DATA
 >   5 .bss          00018e21  c01ba748  c01ba748  000ba748  2**2
 >                   ALLOC
 >   6 .comment      00000ec4  00000000  00000000  000ba748  2**0
 >                   CONTENTS, READONLY
 >   7 .note         00001068  00000ec4  00000ec4  000bb60c  2**0
 >                   CONTENTS, READONLY
 There are obviously 2 non standard ELF sections in the generated object
 file. But first we want to find out what happened to our code in the
 final kernel executable:
 > objdump --disassemble --section=.text vmlinux
 >
 > c017e785 <do_con_write+c1> xorl   %edx,%edx
 > c017e787 <do_con_write+c3> movl   0xc01c7bec,%eax
 > c017e78c <do_con_write+c8> cmpl   $0x18,0x314(%eax)
 > c017e793 <do_con_write+cf> je     c017e79f <do_con_write+db>
 > c017e795 <do_con_write+d1> cmpl   $0xbfffffff,0x40(%esp,1)
 > c017e79d <do_con_write+d9> ja     c017e7a7 <do_con_write+e3>
 > c017e79f <do_con_write+db> movl   %edx,%eax
 > c017e7a1 <do_con_write+dd> movl   0x40(%esp,1),%ebx
 > c017e7a5 <do_con_write+e1> movb   (%ebx),%dl
 > c017e7a7 <do_con_write+e3> movzbl %dl,%esi
 The whole user memory access is reduced to 10 x86 machine instructions.
 The instructions bracketed in the .section directives are no longer
 in the normal execution path. They are located in a different section 
 of the executable file:
 > objdump --disassemble --section=.fixup vmlinux
 > 
 > c0199ff5 <.fixup+10b5> movl   $0xfffffff2,%eax
 > c0199ffa <.fixup+10ba> xorb   %dl,%dl
 > c0199ffc <.fixup+10bc> jmp    c017e7a7 <do_con_write+e3>
 And finally:
 > objdump --full-contents --section=__ex_table vmlinux
 > 
 >  c01aa7c4 93c017c0 e09f19c0 97c017c0 99c017c0  ................
 >  c01aa7d4 f6c217c0 e99f19c0 a5e717c0 f59f19c0  ................
 >  c01aa7e4 080a18c0 01a019c0 0a0a18c0 04a019c0  ................
 or in human readable byte order:
 >  c01aa7c4 c017c093 c0199fe0 c017c097 c017c099  ................
 >  c01aa7d4 c017c2f6 c0199fe9 c017e7a5 c0199ff5  ................
                               ^^^^^^^^^^^^^^^^^
                               this is the interesting part!
 >  c01aa7e4 c0180a08 c019a001 c0180a0a c019a004  ................
 What happened? The assembly directives
 .section .fixup,"ax"
 .section __ex_table,"a"
 told the assembler to move the following code to the specified
 sections in the ELF object file. So the instructions
 3:      movl $-14,%eax
        xorb %dl,%dl
        jmp 2b
 ended up in the .fixup section of the object file and the addresses
        .long 1b,3b
 ended up in the __ex_table section of the object file. 1b and 3b
 are local labels. The local label 1b (1b stands for next label 1 
 backward) is the address of the instruction that might fault, i.e. 
 in our case the address of the label 1 is c017e7a5:
 the original assembly code: > 1:      movb (%ebx),%dl
 and linked in vmlinux     : > c017e7a5 <do_con_write+e1> movb   (%ebx),%dl
 The local label 3 (backwards again) is the address of the code to handle
 the fault, in our case the actual value is c0199ff5:
 the original assembly code: > 3:      movl $-14,%eax
 and linked in vmlinux     : > c0199ff5 <.fixup+10b5> movl   $0xfffffff2,%eax
 The assembly code
 > .section __ex_table,"a"
 >         .align 4
 >         .long 1b,3b
 becomes the value pair
 >  c01aa7d4 c017c2f6 c0199fe9 c017e7a5 c0199ff5  ................
                               ^this is ^this is
                               1b       3b 
 c017e7a5,c0199ff5 in the exception table of the kernel.
 So, what actually happens if a fault from kernel mode with no suitable
 vma occurs?
 1.) access to invalid address:
 > c017e7a5 <do_con_write+e1> movb   (%ebx),%dl
 2.) MMU generates exception
 3.) CPU calls do_page_fault
 4.) do page fault calls search_exception_table (regs->eip == c017e7a5);
 5.) search_exception_table looks up the address c017e7a5 in the
    exception table (i.e. the contents of the ELF section __ex_table) 
    and returns the address of the associated fault handle code c0199ff5.
 6.) do_page_fault modifies its own return address to point to the fault 
    handle code and returns.
 7.) execution continues in the fault handling code.
 8.) 8a) EAX becomes -EFAULT (== -14)
    8b) DL  becomes zero (the value we "read" from user space)
    8c) execution continues at local label 2 (address of the
        instruction immediately after the faulting user access).
 The steps 8a to 8c in a certain way emulate the faulting instruction.
 That's it, mostly. If you look at our example, you might ask why
 we set EAX to -EFAULT in the exception handler code. Well, the
 get_user macro actually returns a value: 0, if the user access was
 successful, -EFAULT on failure. Our original code did not test this
 return value, however the inline assembly code in get_user tries to
 return -EFAULT. GCC selected EAX to return this value.
 NOTE:
 Due to the way that the exception table is built and needs to be ordered,
 only use exceptions for code in the .text section.  Any other section
 will cause the exception table to not be sorted correctly, and the
 exceptions will fail.
 The Framebuffer Console
 =======================
 	The framebuffer console (fbcon), as its name implies, is a text
 console running on top of the framebuffer device. It has the functionality of
 any standard text console driver, such as the VGA console, with the added
 features that can be attributed to the graphical nature of the framebuffer.
 	 In the x86 architecture, the framebuffer console is optional, and
 some even treat it as a toy. For other architectures, it is the only available
 display device, text or graphical.
 	 What are the features of fbcon?  The framebuffer console supports
 high resolutions, varying font types, display rotation, primitive multihead,
 etc. Theoretically, multi-colored fonts, blending, aliasing, and any feature
 made available by the underlying graphics card are also possible.
 A. Configuration
 	The framebuffer console can be enabled by using your favorite kernel
 configuration tool.  It is under Device Drivers->Graphics Support->Support for
 framebuffer devices->Framebuffer Console Support. Select 'y' to compile
 support statically, or 'm' for module support.  The module will be fbcon.
 	In order for fbcon to activate, at least one framebuffer driver is
 required, so choose from any of the numerous drivers available. For x86
 systems, they almost universally have VGA cards, so vga16fb and vesafb will
 always be available. However, using a chipset-specific driver will give you
 more speed and features, such as the ability to change the video mode
 dynamically.
 	To display the penguin logo, choose any logo available in Logo
 Configuration->Boot up logo.
 	Also, you will need to select at least one compiled-in fonts, but if
 you don't do anything, the kernel configuration tool will select one for you,
 usually an 8x16 font.
 GOTCHA: A common bug report is enabling the framebuffer without enabling the
 framebuffer console.  Depending on the driver, you may get a blanked or
 garbled display, but the system still boots to completion.  If you are
 fortunate to have a driver that does not alter the graphics chip, then you
 will still get a VGA console.
 B. Loading
 Possible scenarios:
 1. Driver and fbcon are compiled statically
 	 Usually, fbcon will automatically take over your console. The notable
 	 exception is vesafb.  It needs to be explicitly activated with the
 	 vga= boot option parameter.
 2. Driver is compiled statically, fbcon is compiled as a module
 	 Depending on the driver, you either get a standard console, or a
 	 garbled display, as mentioned above.  To get a framebuffer console,
 	 do a 'modprobe fbcon'.
 3. Driver is compiled as a module, fbcon is compiled statically
 	 You get your standard console.  Once the driver is loaded with
 	 'modprobe xxxfb', fbcon automatically takes over the console with
 	 the possible exception of using the fbcon=map:n option. See below.
 4. Driver and fbcon are compiled as a module.
 	 You can load them in any order. Once both are loaded, fbcon will take
 	 over the console.
 C. Boot options
         The framebuffer console has several, largely unknown, boot options
         that can change its behavior.
 1. fbcon=font:<name>
        Select the initial font to use. The value 'name' can be any of the
        compiled-in fonts: VGA8x16, 7x14, 10x18, VGA8x8, MINI4x6, RomanLarge,
        SUN8x16, SUN12x22, ProFont6x11, Acorn8x8, PEARL8x8.
 	Note, not all drivers can handle font with widths not divisible by 8,
        such as vga16fb.
 2. fbcon=scrollback:<value>[k]
        The scrollback buffer is memory that is used to preserve display
        contents that has already scrolled past your view.  This is accessed
        by using the Shift-PageUp key combination.  The value 'value' is any
        integer. It defaults to 32KB.  The 'k' suffix is optional, and will
        multiply the 'value' by 1024.
 3. fbcon=map:<0123>
        This is an interesting option. It tells which driver gets mapped to
        which console. The value '0123' is a sequence that gets repeated until
        the total length is 64 which is the number of consoles available. In
        the above example, it is expanded to 012301230123... and the mapping
        will be:
 		tty | 1 2 3 4 5 6 7 8 9 ...
 		fb  | 0 1 2 3 0 1 2 3 0 ...
 		('cat /proc/fb' should tell you what the fb numbers are)
 	One side effect that may be useful is using a map value that exceeds
 	the number of loaded fb drivers. For example, if only one driver is
 	available, fb0, adding fbcon=map:1 tells fbcon not to take over the
 	console.
 	Later on, when you want to map the console the to the framebuffer
 	device, you can use the con2fbmap utility.
 4. fbcon=vc:<n1>-<n2>
 	This option tells fbcon to take over only a range of consoles as
 	specified by the values 'n1' and 'n2'. The rest of the consoles
 	outside the given range will still be controlled by the standard
 	console driver.
 	NOTE: For x86 machines, the standard console is the VGA console which
 	is typically located on the same video card.  Thus, the consoles that
 	are controlled by the VGA console will be garbled.
 4. fbcon=rotate:<n>
        This option changes the orientation angle of the console display. The
        value 'n' accepts the following:
 	      0 - normal orientation (0 degree)
 	      1 - clockwise orientation (90 degrees)
 	      2 - upside down orientation (180 degrees)
 	      3 - counterclockwise orientation (270 degrees)
 	The angle can be changed anytime afterwards by 'echoing' the same
 	numbers to any one of the 2 attributes found in
 	 /sys/class/graphics/fbcon
 		rotate     - rotate the display of the active console
 		rotate_all - rotate the display of all consoles
 	Console rotation will only become available if Console Rotation
 	Support is compiled in your kernel.
 	NOTE: This is purely console rotation.  Any other applications that
 	use the framebuffer will remain at their 'normal'orientation.
 	Actually, the underlying fb driver is totally ignorant of console
 	rotation.
 C. Attaching, Detaching and Unloading
 Before going on on how to attach, detach and unload the framebuffer console, an
 illustration of the dependencies may help.
 The console layer, as with most subsystems, needs a driver that interfaces with
 the hardware. Thus, in a VGA console:
 console ---> VGA driver ---> hardware.
 Assuming the VGA driver can be unloaded, one must first unbind the VGA driver
 from the console layer before unloading the driver.  The VGA driver cannot be
 unloaded if it is still bound to the console layer. (See
 Documentation/console/console.txt for more information).
-This is more complicated in the case of the the framebuffer console (fbcon),
+This is more complicated in the case of the framebuffer console (fbcon),
 because fbcon is an intermediate layer between the console and the drivers:
 console ---> fbcon ---> fbdev drivers ---> hardware
 The fbdev drivers cannot be unloaded if it's bound to fbcon, and fbcon cannot
 be unloaded if it's bound to the console layer.
 So to unload the fbdev drivers, one must first unbind fbcon from the console,
 then unbind the fbdev drivers from fbcon.  Fortunately, unbinding fbcon from
 the console layer will automatically unbind framebuffer drivers from
 fbcon. Thus, there is no need to explicitly unbind the fbdev drivers from
 fbcon.
 So, how do we unbind fbcon from the console? Part of the answer is in
 Documentation/console/console.txt. To summarize:
 Echo a value to the bind file that represents the framebuffer console
 driver. So assuming vtcon1 represents fbcon, then:
 echo 1 > sys/class/vtconsole/vtcon1/bind - attach framebuffer console to
                                           console layer
 echo 0 > sys/class/vtconsole/vtcon1/bind - detach framebuffer console from
                                           console layer
 If fbcon is detached from the console layer, your boot console driver (which is
 usually VGA text mode) will take over.  A few drivers (rivafb and i810fb) will
 restore VGA text mode for you.  With the rest, before detaching fbcon, you
 must take a few additional steps to make sure that your VGA text mode is
 restored properly. The following is one of the several methods that you can do:
 1. Download or install vbetool.  This utility is included with most
   distributions nowadays, and is usually part of the suspend/resume tool.
 2. In your kernel configuration, ensure that CONFIG_FRAMEBUFFER_CONSOLE is set
   to 'y' or 'm'. Enable one or more of your favorite framebuffer drivers.
 3. Boot into text mode and as root run:
 	vbetool vbestate save > <vga state file>
 	The above command saves the register contents of your graphics
 	hardware to <vga state file>.  You need to do this step only once as
 	the state file can be reused.
 4. If fbcon is compiled as a module, load fbcon by doing:
       modprobe fbcon
 5. Now to detach fbcon:
       vbetool vbestate restore < <vga state file> && \
       echo 0 > /sys/class/vtconsole/vtcon1/bind
 6. That's it, you're back to VGA mode. And if you compiled fbcon as a module,
   you can unload it by 'rmmod fbcon'
 7. To reattach fbcon:
       echo 1 > /sys/class/vtconsole/vtcon1/bind
 8. Once fbcon is unbound, all drivers registered to the system will also
 become unbound.  This means that fbcon and individual framebuffer drivers
 can be unloaded or reloaded at will. Reloading the drivers or fbcon will
 automatically bind the console, fbcon and the drivers together. Unloading
 all the drivers without unloading fbcon will make it impossible for the
 console to bind fbcon.
 Notes for vesafb users:
 =======================
 Unfortunately, if your bootline includes a vga=xxx parameter that sets the
 hardware in graphics mode, such as when loading vesafb, vgacon will not load.
 Instead, vgacon will replace the default boot console with dummycon, and you
 won't get any display after detaching fbcon. Your machine is still alive, so
 you can reattach vesafb. However, to reattach vesafb, you need to do one of
 the following:
 Variation 1:
    a. Before detaching fbcon, do
       vbetool vbemode save > <vesa state file> # do once for each vesafb mode,
 						# the file can be reused
    b. Detach fbcon as in step 5.
    c. Attach fbcon
        vbetool vbestate restore < <vesa state file> && \
 	echo 1 > /sys/class/vtconsole/vtcon1/bind
 Variation 2:
    a. Before detaching fbcon, do:
 	echo <ID> > /sys/class/tty/console/bind
       vbetool vbemode get
    b. Take note of the mode number
    b. Detach fbcon as in step 5.
    c. Attach fbcon:
       vbetool vbemode set <mode number> && \
       echo 1 > /sys/class/vtconsole/vtcon1/bind
 Samples:
 ========
 Here are 2 sample bash scripts that you can use to bind or unbind the
 framebuffer console driver if you are in an X86 box:
 ---------------------------------------------------------------------------
 #!/bin/bash
 # Unbind fbcon
 # Change this to where your actual vgastate file is located
 # Or Use VGASTATE=$1 to indicate the state file at runtime
 VGASTATE=/tmp/vgastate
 # path to vbetool
 VBETOOL=/usr/local/bin
 for (( i = 0; i < 16; i++))
 do
  if test -x /sys/class/vtconsole/vtcon$i; then
      if [ `cat /sys/class/vtconsole/vtcon$i/name | grep -c "frame buffer"` \
           = 1 ]; then
 	    if test -x $VBETOOL/vbetool; then
 	       echo Unbinding vtcon$i
 	       $VBETOOL/vbetool vbestate restore < $VGASTATE
 	       echo 0 > /sys/class/vtconsole/vtcon$i/bind
 	    fi
      fi
  fi
 done
 ---------------------------------------------------------------------------
 #!/bin/bash
 # Bind fbcon
 for (( i = 0; i < 16; i++))
 do
  if test -x /sys/class/vtconsole/vtcon$i; then
      if [ `cat /sys/class/vtconsole/vtcon$i/name | grep -c "frame buffer"` \
           = 1 ]; then
 	  echo Unbinding vtcon$i
 	  echo 1 > /sys/class/vtconsole/vtcon$i/bind
      fi
  fi
 done
 ---------------------------------------------------------------------------
 --
 Antonino Daplas <adaplas@pol.net>
 	Locking scheme used for directory operations is based on two
 kinds of locks - per-inode (->i_sem) and per-filesystem (->s_vfs_rename_sem).
 	For our purposes all operations fall in 5 classes:
 1) read access.  Locking rules: caller locks directory we are accessing.
 2) object creation.  Locking rules: same as above.
 3) object removal.  Locking rules: caller locks parent, finds victim,
 locks victim and calls the method.
 4) rename() that is _not_ cross-directory.  Locking rules: caller locks
 the parent, finds source and target, if target already exists - locks it
 and then calls the method.
 5) link creation.  Locking rules:
 	* lock parent
 	* check that source is not a directory
 	* lock source
 	* call the method.
 6) cross-directory rename.  The trickiest in the whole bunch.  Locking
 rules:
 	* lock the filesystem
 	* lock parents in "ancestors first" order.
 	* find source and target.
 	* if old parent is equal to or is a descendent of target
 		fail with -ENOTEMPTY
 	* if new parent is equal to or is a descendent of source
 		fail with -ELOOP
 	* if target exists - lock it.
 	* call the method.
 The rules above obviously guarantee that all directories that are going to be
 read, modified or removed by method will be locked by caller.
 If no directory is its own ancestor, the scheme above is deadlock-free.
 Proof:
 	First of all, at any moment we have a partial ordering of the
 objects - A < B iff A is an ancestor of B.
 	That ordering can change.  However, the following is true:
 (1) if object removal or non-cross-directory rename holds lock on A and
    attempts to acquire lock on B, A will remain the parent of B until we
    acquire the lock on B.  (Proof: only cross-directory rename can change
    the parent of object and it would have to lock the parent).
 (2) if cross-directory rename holds the lock on filesystem, order will not
    change until rename acquires all locks.  (Proof: other cross-directory
    renames will be blocked on filesystem lock and we don't start changing
    the order until we had acquired all locks).
 (3) any operation holds at most one lock on non-directory object and
    that lock is acquired after all other locks.  (Proof: see descriptions
    of operations).
 	Now consider the minimal deadlock.  Each process is blocked on
 attempt to acquire some lock and already holds at least one lock.  Let's
 consider the set of contended locks.  First of all, filesystem lock is
 not contended, since any process blocked on it is not holding any locks.
 Thus all processes are blocked on ->i_sem.
 	Non-directory objects are not contended due to (3).  Thus link
 creation can't be a part of deadlock - it can't be blocked on source
 and it means that it doesn't hold any locks.
 	Any contended object is either held by cross-directory rename or
 has a child that is also contended.  Indeed, suppose that it is held by
 operation other than cross-directory rename.  Then the lock this operation
 is blocked on belongs to child of that object due to (1).
 	It means that one of the operations is cross-directory rename.
 Otherwise the set of contended objects would be infinite - each of them
 would have a contended child and we had assumed that no object is its
 own descendent.  Moreover, there is exactly one cross-directory rename
 (see above).
 	Consider the object blocking the cross-directory rename.  One
 of its descendents is locked by cross-directory rename (otherwise we
-would again have an infinite set of of contended objects).  But that
+would again have an infinite set of contended objects).  But that
 means that cross-directory rename is taking locks out of order.  Due
 to (2) the order hadn't changed since we had acquired filesystem lock.
 But locking rules for cross-directory rename guarantee that we do not
 try to acquire lock on descendent before the lock on ancestor.
 Contradiction.  I.e.  deadlock is impossible.  Q.E.D.
 	These operations are guaranteed to avoid loop creation.  Indeed,
 the only operation that could introduce loops is cross-directory rename.
 Since the only new (parent, child) pair added by rename() is (new parent,
 source), such loop would have to contain these objects and the rest of it
 would have to exist before rename().  I.e. at the moment of loop creation
 rename() responsible for that would be holding filesystem lock and new parent
 would have to be equal to or a descendent of source.  But that means that
 new parent had been equal to or a descendent of source since the moment when
 we had acquired filesystem lock and rename() would fail with -ELOOP in that
 case.
 	While this locking scheme works for arbitrary DAGs, it relies on
 ability to check that directory is a descendent of another object.  Current
 implementation assumes that directory graph is a tree.  This assumption is
 also preserved by all operations (cross-directory rename on a tree that would
 not introduce a cycle will leave it a tree and link() fails for directories).
 	Notice that "directory" in the above == "anything that might have
 children", so if we are going to introduce hybrid objects we will need
 either to make sure that link(2) doesn't work for them or to make changes
 in is_subdir() that would make it work even in presence of such beasts.
 File management in the Linux kernel
 -----------------------------------
 This document describes how locking for files (struct file)
 and file descriptor table (struct files) works.
 Up until 2.6.12, the file descriptor table has been protected
 with a lock (files->file_lock) and reference count (files->count).
 ->file_lock protected accesses to all the file related fields
 of the table. ->count was used for sharing the file descriptor
 table between tasks cloned with CLONE_FILES flag. Typically
 this would be the case for posix threads. As with the common
 refcounting model in the kernel, the last task doing
 a put_files_struct() frees the file descriptor (fd) table.
 The files (struct file) themselves are protected using
 reference count (->f_count).
 In the new lock-free model of file descriptor management,
 the reference counting is similar, but the locking is
 based on RCU. The file descriptor table contains multiple
 elements - the fd sets (open_fds and close_on_exec, the
 array of file pointers, the sizes of the sets and the array
 etc.). In order for the updates to appear atomic to
 a lock-free reader, all the elements of the file descriptor
 table are in a separate structure - struct fdtable.
 files_struct contains a pointer to struct fdtable through
 which the actual fd table is accessed. Initially the
 fdtable is embedded in files_struct itself. On a subsequent
 expansion of fdtable, a new fdtable structure is allocated
 and files->fdtab points to the new structure. The fdtable
 structure is freed with RCU and lock-free readers either
 see the old fdtable or the new fdtable making the update
 appear atomic. Here are the locking rules for
 the fdtable structure -
 1. All references to the fdtable must be done through
   the files_fdtable() macro :
 	struct fdtable *fdt;
 	rcu_read_lock();
 	fdt = files_fdtable(files);
 	....
 	if (n <= fdt->max_fds)
 		....
 	...
 	rcu_read_unlock();
   files_fdtable() uses rcu_dereference() macro which takes care of
   the memory barrier requirements for lock-free dereference.
   The fdtable pointer must be read within the read-side
   critical section.
 2. Reading of the fdtable as described above must be protected
   by rcu_read_lock()/rcu_read_unlock().
-3. For any update to the the fd table, files->file_lock must
+3. For any update to the fd table, files->file_lock must
   be held.
 4. To look up the file structure given an fd, a reader
   must use either fcheck() or fcheck_files() APIs. These
   take care of barrier requirements due to lock-free lookup.
   An example :
 	struct file *file;
 	rcu_read_lock();
 	file = fcheck(fd);
 	if (file) {
 		...
 	}
 	....
 	rcu_read_unlock();
 5. Handling of the file structures is special. Since the look-up
   of the fd (fget()/fget_light()) are lock-free, it is possible
   that look-up may race with the last put() operation on the
   file structure. This is avoided using the rcuref APIs
   on ->f_count :
 	rcu_read_lock();
 	file = fcheck_files(files, fd);
 	if (file) {
 		if (rcuref_inc_lf(&file->f_count))
 			*fput_needed = 1;
 		else
 		/* Didn't get the reference, someone's freed */
 			file = NULL;
 	}
 	rcu_read_unlock();
 	....
 	return file;
   rcuref_inc_lf() detects if refcounts is already zero or
   goes to zero during increment. If it does, we fail
   fget()/fget_light().
 6. Since both fdtable and file structures can be looked up
   lock-free, they must be installed using rcu_assign_pointer()
   API. If they are looked up lock-free, rcu_dereference()
   must be used. However it is advisable to use files_fdtable()
   and fcheck()/fcheck_files() which take care of these issues.
 7. While updating, the fdtable pointer must be looked up while
   holding files->file_lock. If ->file_lock is dropped, then
   another thread expand the files thereby creating a new
   fdtable and making the earlier fdtable pointer stale.
   For example :
 	spin_lock(&files->file_lock);
 	fd = locate_fd(files, file, start);
 	if (fd >= 0) {
 		/* locate_fd() may have expanded fdtable, load the ptr */
 		fdt = files_fdtable(files);
 		FD_SET(fd, fdt->open_fds);
 		FD_CLR(fd, fdt->close_on_exec);
 		spin_unlock(&files->file_lock);
 	.....
   Since locate_fd() can drop ->file_lock (and reacquire ->file_lock),
   the fdtable pointer (fdt) must be loaded after locate_fd().
 SPUFS(2)                   Linux Programmer's Manual                  SPUFS(2)
 NAME
       spufs - the SPU file system
 DESCRIPTION
       The SPU file system is used on PowerPC machines that implement the Cell
       Broadband Engine Architecture in order to access Synergistic  Processor
       Units (SPUs).
       The file system provides a name space similar to posix shared memory or
       message queues. Users that have write permissions on  the  file  system
       can use spu_create(2) to establish SPU contexts in the spufs root.
       Every SPU context is represented by a directory containing a predefined
       set of files. These files can be used for manipulating the state of the
       logical SPU. Users can change permissions on those files, but not actu-
       ally add or remove files.
 MOUNT OPTIONS
       uid=<uid>
              set the user owning the mount point, the default is 0 (root).
       gid=<gid>
              set the group owning the mount point, the default is 0 (root).
 FILES
       The files in spufs mostly follow the standard behavior for regular sys-
       tem  calls like read(2) or write(2), but often support only a subset of
       the operations supported on regular file systems. This list details the
       supported  operations  and  the  deviations  from  the behaviour in the
       respective man pages.
       All files that support the read(2) operation also support readv(2)  and
       all  files  that support the write(2) operation also support writev(2).
       All files support the access(2) and stat(2) family of  operations,  but
       only  the  st_mode,  st_nlink,  st_uid and st_gid fields of struct stat
       contain reliable information.
       All files support the chmod(2)/fchmod(2) and chown(2)/fchown(2)  opera-
       tions,  but  will  not be able to grant permissions that contradict the
       possible operations, e.g. read access on the wbox file.
       The current set of files is:
   /mem
       the contents of the local storage memory  of  the  SPU.   This  can  be
       accessed  like  a regular shared memory file and contains both code and
       data in the address space of the SPU.  The possible  operations  on  an
       open mem file are:
       read(2), pread(2), write(2), pwrite(2), lseek(2)
              These  operate  as  documented, with the exception that seek(2),
              write(2) and pwrite(2) are not supported beyond the end  of  the
              file. The file size is the size of the local storage of the SPU,
              which normally is 256 kilobytes.
       mmap(2)
              Mapping mem into the process address space gives access  to  the
              SPU  local  storage  within  the  process  address  space.  Only
              MAP_SHARED mappings are allowed.
   /mbox
       The first SPU to CPU communication mailbox. This file is read-only  and
       can  be  read  in  units of 32 bits.  The file can only be used in non-
       blocking mode and it even poll() will not block on  it.   The  possible
       operations on an open mbox file are:
       read(2)
              If  a  count smaller than four is requested, read returns -1 and
              sets errno to EINVAL.  If there is no data available in the mail
              box,  the  return  value  is set to -1 and errno becomes EAGAIN.
              When data has been read successfully, four bytes are  placed  in
              the data buffer and the value four is returned.
   /ibox
       The  second  SPU  to CPU communication mailbox. This file is similar to
       the first mailbox file, but can be read in blocking I/O mode,  and  the
       poll  family of system calls can be used to wait for it.  The  possible
       operations on an open ibox file are:
       read(2)
              If a count smaller than four is requested, read returns  -1  and
              sets errno to EINVAL.  If there is no data available in the mail
              box and the file descriptor has been opened with O_NONBLOCK, the
              return value is set to -1 and errno becomes EAGAIN.
              If  there  is  no  data  available  in the mail box and the file
              descriptor has been opened without  O_NONBLOCK,  the  call  will
              block  until  the  SPU  writes to its interrupt mailbox channel.
              When data has been read successfully, four bytes are  placed  in
              the data buffer and the value four is returned.
       poll(2)
              Poll  on  the  ibox  file returns (POLLIN | POLLRDNORM) whenever
              data is available for reading.
   /wbox
-       The CPU to SPU communation mailbox. It is write-only can can be written
+       The CPU to SPU communation mailbox. It is write-only and can be written
       in  units  of  32  bits. If the mailbox is full, write() will block and
       poll can be used to wait for it becoming  empty  again.   The  possible
       operations  on  an open wbox file are: write(2) If a count smaller than
       four is requested, write returns -1 and sets errno to EINVAL.  If there
       is  no space available in the mail box and the file descriptor has been
       opened with O_NONBLOCK, the return value is set to -1 and errno becomes
       EAGAIN.
       If  there is no space available in the mail box and the file descriptor
       has been opened without O_NONBLOCK, the call will block until  the  SPU
       reads  from  its PPE mailbox channel.  When data has been read success-
       fully, four bytes are placed in the data buffer and the value  four  is
       returned.
       poll(2)
              Poll  on  the  ibox file returns (POLLOUT | POLLWRNORM) whenever
              space is available for writing.
   /mbox_stat
   /ibox_stat
   /wbox_stat
       Read-only files that contain the length of the current queue, i.e.  how
       many  words  can  be  read  from  mbox or ibox or how many words can be
       written to wbox without blocking.  The files can be read only in 4-byte
       units  and  return  a  big-endian  binary integer number.  The possible
       operations on an open *box_stat file are:
       read(2)
              If a count smaller than four is requested, read returns  -1  and
              sets errno to EINVAL.  Otherwise, a four byte value is placed in
              the data buffer, containing the number of elements that  can  be
              read  from  (for  mbox_stat  and  ibox_stat)  or written to (for
              wbox_stat) the respective mail box without blocking or resulting
              in EAGAIN.
   /npc
   /decr
   /decr_status
   /spu_tag_mask
   /event_mask
   /srr0
       Internal  registers  of  the SPU. The representation is an ASCII string
       with the numeric value of the next instruction to  be  executed.  These
       can  be  used in read/write mode for debugging, but normal operation of
       programs should not rely on them because access to any of  them  except
       npc requires an SPU context save and is therefore very inefficient.
       The contents of these files are:
       npc                 Next Program Counter
       decr                SPU Decrementer
       decr_status         Decrementer Status
       spu_tag_mask        MFC tag mask for SPU DMA
       event_mask          Event mask for SPU interrupts
       srr0                Interrupt Return address register
       The   possible   operations   on   an   open  npc,  decr,  decr_status,
       spu_tag_mask, event_mask or srr0 file are:
       read(2)
              When the count supplied to the read call  is  shorter  than  the
              required  length for the pointer value plus a newline character,
              subsequent reads from the same file descriptor  will  result  in
              completing  the string, regardless of changes to the register by
              a running SPU task.  When a complete string has been  read,  all
              subsequent read operations will return zero bytes and a new file
              descriptor needs to be opened to read the value again.
       write(2)
              A write operation on the file results in setting the register to
              the  value  given  in  the string. The string is parsed from the
              beginning to the first non-numeric character or the end  of  the
              buffer.  Subsequent writes to the same file descriptor overwrite
              the previous setting.
   /fpcr
       This file gives access to the Floating Point Status and Control  Regis-
       ter as a four byte long file. The operations on the fpcr file are:
       read(2)
              If  a  count smaller than four is requested, read returns -1 and
              sets errno to EINVAL.  Otherwise, a four byte value is placed in
              the data buffer, containing the current value of the fpcr regis-
              ter.
       write(2)
              If a count smaller than four is requested, write returns -1  and
              sets  errno  to  EINVAL.  Otherwise, a four byte value is copied
              from the data buffer, updating the value of the fpcr register.
   /signal1
   /signal2
       The two signal notification channels of an SPU.  These  are  read-write
       files  that  operate  on  a 32 bit word.  Writing to one of these files
       triggers an interrupt on the SPU. The  value  writting  to  the  signal
       files can be read from the SPU through a channel read or from host user
       space through the file.  After the value has been read by the  SPU,  it
       is  reset  to zero.  The possible operations on an open signal1 or sig-
       nal2 file are:
       read(2)
              If a count smaller than four is requested, read returns  -1  and
              sets errno to EINVAL.  Otherwise, a four byte value is placed in
              the data buffer, containing the current value of  the  specified
              signal notification register.
       write(2)
              If  a count smaller than four is requested, write returns -1 and
              sets errno to EINVAL.  Otherwise, a four byte  value  is  copied
              from the data buffer, updating the value of the specified signal
              notification register.  The signal  notification  register  will
              either be replaced with the input data or will be updated to the
              bitwise OR or the old value and the input data, depending on the
              contents  of  the  signal1_type,  or  signal2_type respectively,
              file.
   /signal1_type
   /signal2_type
       These two files change the behavior of the signal1 and signal2  notifi-
       cation  files.  The  contain  a numerical ASCII string which is read as
       either "1" or "0".  In mode 0 (overwrite), the  hardware  replaces  the
       contents of the signal channel with the data that is written to it.  in
       mode 1 (logical OR), the hardware accumulates the bits that are  subse-
       quently written to it.  The possible operations on an open signal1_type
       or signal2_type file are:
       read(2)
              When the count supplied to the read call  is  shorter  than  the
              required  length  for the digit plus a newline character, subse-
              quent reads from the same file descriptor will  result  in  com-
              pleting  the  string.  When a complete string has been read, all
              subsequent read operations will return zero bytes and a new file
              descriptor needs to be opened to read the value again.
       write(2)
              A write operation on the file results in setting the register to
              the value given in the string. The string  is  parsed  from  the
              beginning  to  the first non-numeric character or the end of the
              buffer.  Subsequent writes to the same file descriptor overwrite
              the previous setting.
 EXAMPLES
       /etc/fstab entry
              none      /spu      spufs     gid=spu   0    0
 AUTHORS
       Arnd  Bergmann  <arndb@de.ibm.com>,  Mark  Nutter <mnutter@us.ibm.com>,
       Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
 SEE ALSO
       capabilities(7), close(2), spu_create(2), spu_run(2), spufs(7)
 Linux                             2005-09-28                          SPUFS(2)
 ------------------------------------------------------------------------------
 SPU_RUN(2)                 Linux Programmer's Manual                SPU_RUN(2)
 NAME
       spu_run - execute an spu context
 SYNOPSIS
       #include <sys/spu.h>
       int spu_run(int fd, unsigned int *npc, unsigned int *event);
 DESCRIPTION
       The  spu_run system call is used on PowerPC machines that implement the
       Cell Broadband Engine Architecture in order to access Synergistic  Pro-
       cessor  Units  (SPUs).  It  uses the fd that was returned from spu_cre-
       ate(2) to address a specific SPU context. When the context gets  sched-
       uled  to a physical SPU, it starts execution at the instruction pointer
       passed in npc.
       Execution of SPU code happens synchronously, meaning that spu_run  does
       not  return  while the SPU is still running. If there is a need to exe-
       cute SPU code in parallel with other code on either  the  main  CPU  or
       other  SPUs,  you  need to create a new thread of execution first, e.g.
       using the pthread_create(3) call.
       When spu_run returns, the current value of the SPU instruction  pointer
       is  written back to npc, so you can call spu_run again without updating
       the pointers.
       event can be a NULL pointer or point to an extended  status  code  that
       gets  filled  when spu_run returns. It can be one of the following con-
       stants:
       SPE_EVENT_DMA_ALIGNMENT
              A DMA alignment error
       SPE_EVENT_SPE_DATA_SEGMENT
              A DMA segmentation error
       SPE_EVENT_SPE_DATA_STORAGE
              A DMA storage error
       If NULL is passed as the event argument, these errors will result in  a
       signal delivered to the calling process.
 RETURN VALUE
       spu_run  returns the value of the spu_status register or -1 to indicate
       an error and set errno to one of the error  codes  listed  below.   The
       spu_status  register  value  contains  a  bit  mask of status codes and
       optionally a 14 bit code returned from the stop-and-signal  instruction
       on the SPU. The bit masks for the status codes are:
       0x02   SPU was stopped by stop-and-signal.
       0x04   SPU was stopped by halt.
       0x08   SPU is waiting for a channel.
       0x10   SPU is in single-step mode.
       0x20   SPU has tried to execute an invalid instruction.
       0x40   SPU has tried to access an invalid channel.
       0x3fff0000
              The  bits  masked with this value contain the code returned from
              stop-and-signal.
       There are always one or more of the lower eight bits set  or  an  error
       code is returned from spu_run.
 ERRORS
       EAGAIN or EWOULDBLOCK
              fd is in non-blocking mode and spu_run would block.
       EBADF  fd is not a valid file descriptor.
       EFAULT npc is not a valid pointer or status is neither NULL nor a valid
              pointer.
       EINTR  A signal occurred while spu_run was in progress.  The npc  value
              has  been updated to the new program counter value if necessary.
       EINVAL fd is not a file descriptor returned from spu_create(2).
       ENOMEM Insufficient memory was available to handle a page fault result-
              ing from an MFC direct memory access.
       ENOSYS the functionality is not provided by the current system, because
              either the hardware does not provide SPUs or the spufs module is
              not loaded.
 NOTES
       spu_run  is  meant  to  be  used  from  libraries that implement a more
       abstract interface to SPUs, not to be used from  regular  applications.
       See  http://www.bsc.es/projects/deepcomputing/linuxoncell/ for the rec-
       ommended libraries.
 CONFORMING TO
       This call is Linux specific and only implemented by the ppc64 architec-
       ture. Programs using this system call are not portable.
 BUGS
       The code does not yet fully implement all features lined out here.
 AUTHOR
       Arnd Bergmann <arndb@de.ibm.com>
 SEE ALSO
       capabilities(7), close(2), spu_create(2), spufs(7)
 Linux                             2005-09-28                        SPU_RUN(2)
 ------------------------------------------------------------------------------
 SPU_CREATE(2)              Linux Programmer's Manual             SPU_CREATE(2)
 NAME
       spu_create - create a new spu context
 SYNOPSIS
       #include <sys/types.h>
       #include <sys/spu.h>
       int spu_create(const char *pathname, int flags, mode_t mode);
 DESCRIPTION
       The  spu_create  system call is used on PowerPC machines that implement
       the Cell Broadband Engine Architecture in order to  access  Synergistic
       Processor  Units (SPUs). It creates a new logical context for an SPU in
       pathname and returns a handle to associated  with  it.   pathname  must
       point  to  a  non-existing directory in the mount point of the SPU file
       system (spufs).  When spu_create is successful, a directory  gets  cre-
       ated on pathname and it is populated with files.
       The  returned  file  handle can only be passed to spu_run(2) or closed,
       other operations are not defined on it. When it is closed, all  associ-
       ated  directory entries in spufs are removed. When the last file handle
       pointing either inside  of  the  context  directory  or  to  this  file
       descriptor is closed, the logical SPU context is destroyed.
       The  parameter flags can be zero or any bitwise or'd combination of the
       following constants:
       SPU_RAWIO
              Allow mapping of some of the hardware registers of the SPU  into
              user space. This flag requires the CAP_SYS_RAWIO capability, see
              capabilities(7).
       The mode parameter specifies the permissions used for creating the  new
       directory  in  spufs.   mode is modified with the user's umask(2) value
       and then used for both the directory and the files contained in it. The
       file permissions mask out some more bits of mode because they typically
       support only read or write access. See stat(2) for a full list  of  the
       possible mode values.
 RETURN VALUE
       spu_create  returns a new file descriptor. It may return -1 to indicate
       an error condition and set errno to  one  of  the  error  codes  listed
       below.
 ERRORS
       EACCESS
              The  current  user does not have write access on the spufs mount
              point.
       EEXIST An SPU context already exists at the given path name.
       EFAULT pathname is not a valid string pointer in  the  current  address
              space.
       EINVAL pathname is not a directory in the spufs mount point.
       ELOOP  Too many symlinks were found while resolving pathname.
       EMFILE The process has reached its maximum open file limit.
       ENAMETOOLONG
              pathname was too long.
       ENFILE The system has reached the global open file limit.
       ENOENT Part of pathname could not be resolved.
       ENOMEM The kernel could not allocate all resources required.
       ENOSPC There  are  not  enough  SPU resources available to create a new
              context or the user specific limit for the number  of  SPU  con-
              texts has been reached.
       ENOSYS the functionality is not provided by the current system, because
              either the hardware does not provide SPUs or the spufs module is
              not loaded.
       ENOTDIR
              A part of pathname is not a directory.
 NOTES
       spu_create  is  meant  to  be used from libraries that implement a more
       abstract interface to SPUs, not to be used from  regular  applications.
       See  http://www.bsc.es/projects/deepcomputing/linuxoncell/ for the rec-
       ommended libraries.
 FILES
       pathname must point to a location beneath the mount point of spufs.  By
       convention, it gets mounted in /spu.
 CONFORMING TO
       This call is Linux specific and only implemented by the ppc64 architec-
       ture. Programs using this system call are not portable.
 BUGS
       The code does not yet fully implement all features lined out here.
 AUTHOR
       Arnd Bergmann <arndb@de.ibm.com>
 SEE ALSO
       capabilities(7), close(2), spu_run(2), spufs(7)
 Linux                             2005-09-28                     SPU_CREATE(2)
 Tmpfs is a file system which keeps all files in virtual memory.
 Everything in tmpfs is temporary in the sense that no files will be
 created on your hard drive. If you unmount a tmpfs instance,
 everything stored therein is lost.
 tmpfs puts everything into the kernel internal caches and grows and
 shrinks to accommodate the files it contains and is able to swap
 unneeded pages out to swap space. It has maximum size limits which can
 be adjusted on the fly via 'mount -o remount ...'
 If you compare it to ramfs (which was the template to create tmpfs)
 you gain swapping and limit checking. Another similar thing is the RAM
 disk (/dev/ram*), which simulates a fixed size hard disk in physical
 RAM, where you have to create an ordinary filesystem on top. Ramdisks
 cannot swap and you do not have the possibility to resize them. 
 Since tmpfs lives completely in the page cache and on swap, all tmpfs
 pages currently in memory will show up as cached. It will not show up
 as shared or something like that. Further on you can check the actual
 RAM+swap use of a tmpfs instance with df(1) and du(1).
 tmpfs has the following uses:
 1) There is always a kernel internal mount which you will not see at
   all. This is used for shared anonymous mappings and SYSV shared
   memory. 
   This mount does not depend on CONFIG_TMPFS. If CONFIG_TMPFS is not
   set, the user visible part of tmpfs is not build. But the internal
   mechanisms are always present.
 2) glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for
   POSIX shared memory (shm_open, shm_unlink). Adding the following
   line to /etc/fstab should take care of this:
 	tmpfs	/dev/shm	tmpfs	defaults	0 0
   Remember to create the directory that you intend to mount tmpfs on
   if necessary.
   This mount is _not_ needed for SYSV shared memory. The internal
   mount is used for that. (In the 2.3 kernel versions it was
   necessary to mount the predecessor of tmpfs (shm fs) to use SYSV
   shared memory)
 3) Some people (including me) find it very convenient to mount it
   e.g. on /tmp and /var/tmp and have a big swap partition. And now
   loop mounts of tmpfs files do work, so mkinitrd shipped by most
   distributions should succeed with a tmpfs /tmp.
 4) And probably a lot more I do not know about :-)
 tmpfs has three mount options for sizing:
 size:      The limit of allocated bytes for this tmpfs instance. The 
           default is half of your physical RAM without swap. If you
           oversize your tmpfs instances the machine will deadlock
           since the OOM handler will not be able to free that memory.
 nr_blocks: The same as size, but in blocks of PAGE_CACHE_SIZE.
 nr_inodes: The maximum number of inodes for this instance. The default
           is half of the number of your physical RAM pages, or (on a
-           a machine with highmem) the number of lowmem RAM pages,
+           machine with highmem) the number of lowmem RAM pages,
           whichever is the lower.
 These parameters accept a suffix k, m or g for kilo, mega and giga and
 can be changed on remount.  The size parameter also accepts a suffix %
 to limit this tmpfs instance to that percentage of your physical RAM:
 the default, when neither size nor nr_blocks is specified, is size=50%
 If nr_blocks=0 (or size=0), blocks will not be limited in that instance;
 if nr_inodes=0, inodes will not be limited.  It is generally unwise to
 mount with such options, since it allows any user with write access to
 use up all the memory on the machine; but enhances the scalability of
 that instance in a system with many cpus making intensive use of it.
 tmpfs has a mount option to set the NUMA memory allocation policy for
 all files in that instance (if CONFIG_NUMA is enabled) - which can be
 adjusted on the fly via 'mount -o remount ...'
 mpol=default             prefers to allocate memory from the local node
 mpol=prefer:Node         prefers to allocate memory from the given Node
 mpol=bind:NodeList       allocates memory only from nodes in NodeList
 mpol=interleave          prefers to allocate from each node in turn
 mpol=interleave:NodeList allocates from each node of NodeList in turn
 NodeList format is a comma-separated list of decimal numbers and ranges,
 a range being two hyphen-separated decimal numbers, the smallest and
 largest node numbers in the range.  For example, mpol=bind:0-3,5,7,9-15
 Note that trying to mount a tmpfs with an mpol option will fail if the
 running kernel does not support NUMA; and will fail if its nodelist
 specifies a node >= MAX_NUMNODES.  If your system relies on that tmpfs
 being mounted, but from time to time runs a kernel built without NUMA
 capability (perhaps a safe recovery kernel), or configured to support
 fewer nodes, then it is advisable to omit the mpol option from automatic
 mount options.  It can be added later, when the tmpfs is already mounted
 on MountPoint, by 'mount -o remount,mpol=Policy:NodeList MountPoint'.
 To specify the initial root directory you can use the following mount
 options:
 mode:	The permissions as an octal number
 uid:	The user id 
 gid:	The group id
 These options do not have any effect on remount. You can change these
 parameters with chmod(1), chown(1) and chgrp(1) on a mounted filesystem.
 So 'mount -t tmpfs -o size=10G,nr_inodes=10k,mode=700 tmpfs /mytmpfs'
 will give you tmpfs instance on /mytmpfs which can allocate 10GB
 RAM/SWAP in 10240 inodes and it is only accessible by root.
 Author:
   Christoph Rohland <cr@sap.com>, 1.12.01
 Updated:
   Hugh Dickins <hugh@veritas.com>, 19 February 2006
 USING VFAT
 ----------------------------------------------------------------------
 To use the vfat filesystem, use the filesystem type 'vfat'.  i.e.
  mount -t vfat /dev/fd0 /mnt
 No special partition formatter is required.  mkdosfs will work fine
 if you want to format from within Linux.
 VFAT MOUNT OPTIONS
 ----------------------------------------------------------------------
 umask=###     -- The permission mask (for files and directories, see umask(1)).
                 The default is the umask of current process.
 dmask=###     -- The permission mask for the directory.
                 The default is the umask of current process.
 fmask=###     -- The permission mask for files.
                 The default is the umask of current process.
 codepage=###  -- Sets the codepage number for converting to shortname
 		 characters on FAT filesystem.
 		 By default, FAT_DEFAULT_CODEPAGE setting is used.
 iocharset=name -- Character set to use for converting between the
 		 encoding is used for user visible filename and 16 bit
 		 Unicode characters. Long filenames are stored on disk
 		 in Unicode format, but Unix for the most part doesn't
 		 know how to deal with Unicode.
 		 By default, FAT_DEFAULT_IOCHARSET setting is used.
 		 There is also an option of doing UTF-8 translations
 		 with the utf8 option.
 		 NOTE: "iocharset=utf8" is not recommended. If unsure,
 		 you should consider the following option instead.
 utf8=<bool>   -- UTF-8 is the filesystem safe version of Unicode that
-		 is used by the console.  It can be be enabled for the
+		 is used by the console.  It can be enabled for the
 		 filesystem with this option. If 'uni_xlate' gets set,
 		 UTF-8 gets disabled.
 uni_xlate=<bool> -- Translate unhandled Unicode characters to special
 		 escaped sequences.  This would let you backup and
 		 restore filenames that are created with any Unicode
 		 characters.  Until Linux supports Unicode for real,
 		 this gives you an alternative.  Without this option,
 		 a '?' is used when no translation is possible.  The
 		 escape character is ':' because it is otherwise
 		 illegal on the vfat filesystem.  The escape sequence
 		 that gets used is ':' and the four digits of hexadecimal
 		 unicode.
 nonumtail=<bool> -- When creating 8.3 aliases, normally the alias will
                 end in '~1' or tilde followed by some number.  If this
                 option is set, then if the filename is 
                 "longfilename.txt" and "longfile.txt" does not
                 currently exist in the directory, 'longfile.txt' will
                 be the short alias instead of 'longfi~1.txt'. 
 quiet         -- Stops printing certain warning messages.
 check=s|r|n   -- Case sensitivity checking setting.
                 s: strict, case sensitive
                 r: relaxed, case insensitive
                 n: normal, default setting, currently case insensitive
 shortname=lower|win95|winnt|mixed
 	      -- Shortname display/create setting.
 		 lower: convert to lowercase for display,
 			emulate the Windows 95 rule for create.
 		 win95: emulate the Windows 95 rule for display/create.
 		 winnt: emulate the Windows NT rule for display/create.
 		 mixed: emulate the Windows NT rule for display,
 			emulate the Windows 95 rule for create.
 		 Default setting is `lower'.
 <bool>: 0,1,yes,no,true,false
 TODO
 ----------------------------------------------------------------------
 * Need to get rid of the raw scanning stuff.  Instead, always use
  a get next directory entry approach.  The only thing left that uses
  raw scanning is the directory renaming code.
 POSSIBLE PROBLEMS
 ----------------------------------------------------------------------
 * vfat_valid_longname does not properly checked reserved names.
 * When a volume name is the same as a directory name in the root
  directory of the filesystem, the directory name sometimes shows
  up as an empty file.
 * autoconv option does not work correctly.
 BUG REPORTS
 ----------------------------------------------------------------------
 If you have trouble with the VFAT filesystem, mail bug reports to
 chaffee@bmrc.cs.berkeley.edu.  Please specify the filename
 and the operation that gave you trouble.
 TEST SUITE
 ----------------------------------------------------------------------
 If you plan to make any modifications to the vfat filesystem, please
 get the test suite that comes with the vfat distribution at
  http://bmrc.berkeley.edu/people/chaffee/vfat.html
 This tests quite a few parts of the vfat filesystem and additional
 tests for new features or untested features would be appreciated.
 NOTES ON THE STRUCTURE OF THE VFAT FILESYSTEM
 ----------------------------------------------------------------------
 (This documentation was provided by Galen C. Hunt <gchunt@cs.rochester.edu>
 and lightly annotated by Gordon Chaffee).
 This document presents a very rough, technical overview of my
 knowledge of the extended FAT file system used in Windows NT 3.5 and
 Windows 95.  I don't guarantee that any of the following is correct,
 but it appears to be so.
 The extended FAT file system is almost identical to the FAT
 file system used in DOS versions up to and including 6.223410239847
 :-).  The significant change has been the addition of long file names.
 These names support up to 255 characters including spaces and lower
 case characters as opposed to the traditional 8.3 short names.
 Here is the description of the traditional FAT entry in the current
 Windows 95 filesystem:
        struct directory { // Short 8.3 names 
                unsigned char name[8];          // file name 
                unsigned char ext[3];           // file extension 
                unsigned char attr;             // attribute byte 
 		unsigned char lcase;		// Case for base and extension
 		unsigned char ctime_ms;		// Creation time, milliseconds
 		unsigned char ctime[2];		// Creation time
 		unsigned char cdate[2];		// Creation date
 		unsigned char adate[2];		// Last access date
 		unsigned char reserved[2];	// reserved values (ignored) 
                unsigned char time[2];          // time stamp 
                unsigned char date[2];          // date stamp 
                unsigned char start[2];         // starting cluster number 
                unsigned char size[4];          // size of the file 
        };
 The lcase field specifies if the base and/or the extension of an 8.3
 name should be capitalized.  This field does not seem to be used by
 Windows 95 but it is used by Windows NT.  The case of filenames is not
 completely compatible from Windows NT to Windows 95.  It is not completely
 compatible in the reverse direction, however.  Filenames that fit in
 the 8.3 namespace and are written on Windows NT to be lowercase will
 show up as uppercase on Windows 95.
 Note that the "start" and "size" values are actually little
 endian integer values.  The descriptions of the fields in this
 structure are public knowledge and can be found elsewhere.
 With the extended FAT system, Microsoft has inserted extra
 directory entries for any files with extended names.  (Any name which
 legally fits within the old 8.3 encoding scheme does not have extra
 entries.)  I call these extra entries slots.  Basically, a slot is a
 specially formatted directory entry which holds up to 13 characters of
 a file's extended name.  Think of slots as additional labeling for the
 directory entry of the file to which they correspond.  Microsoft
 prefers to refer to the 8.3 entry for a file as its alias and the
 extended slot directory entries as the file name. 
 The C structure for a slot directory entry follows:
        struct slot { // Up to 13 characters of a long name 
                unsigned char id;               // sequence number for slot 
                unsigned char name0_4[10];      // first 5 characters in name 
                unsigned char attr;             // attribute byte
                unsigned char reserved;         // always 0 
                unsigned char alias_checksum;   // checksum for 8.3 alias 
                unsigned char name5_10[12];     // 6 more characters in name
                unsigned char start[2];         // starting cluster number
                unsigned char name11_12[4];     // last 2 characters in name
        };
 If the layout of the slots looks a little odd, it's only
 because of Microsoft's efforts to maintain compatibility with old
 software.  The slots must be disguised to prevent old software from
 panicking.  To this end, a number of measures are taken:
        1) The attribute byte for a slot directory entry is always set
           to 0x0f.  This corresponds to an old directory entry with
           attributes of "hidden", "system", "read-only", and "volume
           label".  Most old software will ignore any directory
           entries with the "volume label" bit set.  Real volume label
           entries don't have the other three bits set.
        2) The starting cluster is always set to 0, an impossible
           value for a DOS file.
 Because the extended FAT system is backward compatible, it is
 possible for old software to modify directory entries.  Measures must
 be taken to ensure the validity of slots.  An extended FAT system can
 verify that a slot does in fact belong to an 8.3 directory entry by
 the following:
        1) Positioning.  Slots for a file always immediately proceed
           their corresponding 8.3 directory entry.  In addition, each
           slot has an id which marks its order in the extended file
           name.  Here is a very abbreviated view of an 8.3 directory
           entry and its corresponding long name slots for the file
           "My Big File.Extension which is long":
                <proceeding files...>
                <slot #3, id = 0x43, characters = "h is long">
                <slot #2, id = 0x02, characters = "xtension whic">
                <slot #1, id = 0x01, characters = "My Big File.E">
                <directory entry, name = "MYBIGFIL.EXT">
           Note that the slots are stored from last to first.  Slots
           are numbered from 1 to N.  The Nth slot is or'ed with 0x40
           to mark it as the last one.
        2) Checksum.  Each slot has an "alias_checksum" value.  The
           checksum is calculated from the 8.3 name using the
           following algorithm:
                for (sum = i = 0; i < 11; i++) {
                        sum = (((sum&1)<<7)|((sum&0xfe)>>1)) + name[i]
                }
 	3) If there is free space in the final slot, a Unicode NULL (0x0000) 
 	   is stored after the final character.  After that, all unused 
 	   characters in the final slot are set to Unicode 0xFFFF.
 Finally, note that the extended name is stored in Unicode.  Each Unicode
 character takes two bytes.
 	      Overview of the Linux Virtual File System
 	Original author: Richard Gooch <rgooch@atnf.csiro.au>
 		  Last updated on October 28, 2005
  Copyright (C) 1999 Richard Gooch
  Copyright (C) 2005 Pekka Enberg
  This file is released under the GPLv2.
 Introduction
 ============
 The Virtual File System (also known as the Virtual Filesystem Switch)
 is the software layer in the kernel that provides the filesystem
 interface to userspace programs. It also provides an abstraction
 within the kernel which allows different filesystem implementations to
 coexist.
 VFS system calls open(2), stat(2), read(2), write(2), chmod(2) and so
 on are called from a process context. Filesystem locking is described
 in the document Documentation/filesystems/Locking.
 Directory Entry Cache (dcache)
 ------------------------------
 The VFS implements the open(2), stat(2), chmod(2), and similar system
 calls. The pathname argument that is passed to them is used by the VFS
 to search through the directory entry cache (also known as the dentry
 cache or dcache). This provides a very fast look-up mechanism to
 translate a pathname (filename) into a specific dentry. Dentries live
 in RAM and are never saved to disc: they exist only for performance.
 The dentry cache is meant to be a view into your entire filespace. As
 most computers cannot fit all dentries in the RAM at the same time,
 some bits of the cache are missing. In order to resolve your pathname
 into a dentry, the VFS may have to resort to creating dentries along
 the way, and then loading the inode. This is done by looking up the
 inode.
 The Inode Object
 ----------------
 An individual dentry usually has a pointer to an inode. Inodes are
 filesystem objects such as regular files, directories, FIFOs and other
 beasts.  They live either on the disc (for block device filesystems)
 or in the memory (for pseudo filesystems). Inodes that live on the
 disc are copied into the memory when required and changes to the inode
 are written back to disc. A single inode can be pointed to by multiple
 dentries (hard links, for example, do this).
 To look up an inode requires that the VFS calls the lookup() method of
 the parent directory inode. This method is installed by the specific
 filesystem implementation that the inode lives in. Once the VFS has
 the required dentry (and hence the inode), we can do all those boring
 things like open(2) the file, or stat(2) it to peek at the inode
 data. The stat(2) operation is fairly simple: once the VFS has the
 dentry, it peeks at the inode data and passes some of it back to
 userspace.
 The File Object
 ---------------
 Opening a file requires another operation: allocation of a file
 structure (this is the kernel-side implementation of file
 descriptors). The freshly allocated file structure is initialized with
 a pointer to the dentry and a set of file operation member functions.
 These are taken from the inode data. The open() file method is then
 called so the specific filesystem implementation can do it's work. You
 can see that this is another switch performed by the VFS. The file
 structure is placed into the file descriptor table for the process.
 Reading, writing and closing files (and other assorted VFS operations)
 is done by using the userspace file descriptor to grab the appropriate
 file structure, and then calling the required file structure method to
 do whatever is required. For as long as the file is open, it keeps the
 dentry in use, which in turn means that the VFS inode is still in use.
 Registering and Mounting a Filesystem
 =====================================
 To register and unregister a filesystem, use the following API
 functions:
   #include <linux/fs.h>
   extern int register_filesystem(struct file_system_type *);
   extern int unregister_filesystem(struct file_system_type *);
 The passed struct file_system_type describes your filesystem. When a
 request is made to mount a device onto a directory in your filespace,
 the VFS will call the appropriate get_sb() method for the specific
 filesystem. The dentry for the mount point will then be updated to
 point to the root inode for the new filesystem.
 You can see all filesystems that are registered to the kernel in the
 file /proc/filesystems.
 struct file_system_type
 -----------------------
 This describes the filesystem. As of kernel 2.6.13, the following
 members are defined:
 struct file_system_type {
 	const char *name;
 	int fs_flags;
        int (*get_sb) (struct file_system_type *, int,
                       const char *, void *, struct vfsmount *);
        void (*kill_sb) (struct super_block *);
        struct module *owner;
        struct file_system_type * next;
        struct list_head fs_supers;
 };
  name: the name of the filesystem type, such as "ext2", "iso9660",
 	"msdos" and so on
  fs_flags: various flags (i.e. FS_REQUIRES_DEV, FS_NO_DCACHE, etc.)
  get_sb: the method to call when a new instance of this
 	filesystem should be mounted
  kill_sb: the method to call when an instance of this filesystem
 	should be unmounted
  owner: for internal VFS use: you should initialize this to THIS_MODULE in
  	most cases.
  next: for internal VFS use: you should initialize this to NULL
 The get_sb() method has the following arguments:
  struct super_block *sb: the superblock structure. This is partially
 	initialized by the VFS and the rest must be initialized by the
 	get_sb() method
  int flags: mount flags
  const char *dev_name: the device name we are mounting.
  void *data: arbitrary mount options, usually comes as an ASCII
 	string
  int silent: whether or not to be silent on error
 The get_sb() method must determine if the block device specified
 in the superblock contains a filesystem of the type the method
 supports. On success the method returns the superblock pointer, on
 failure it returns NULL.
 The most interesting member of the superblock structure that the
 get_sb() method fills in is the "s_op" field. This is a pointer to
 a "struct super_operations" which describes the next level of the
 filesystem implementation.
 Usually, a filesystem uses one of the generic get_sb() implementations
 and provides a fill_super() method instead. The generic methods are:
  get_sb_bdev: mount a filesystem residing on a block device
  get_sb_nodev: mount a filesystem that is not backed by a device
  get_sb_single: mount a filesystem which shares the instance between
  	all mounts
 A fill_super() method implementation has the following arguments:
  struct super_block *sb: the superblock structure. The method fill_super()
  	must initialize this properly.
  void *data: arbitrary mount options, usually comes as an ASCII
 	string
  int silent: whether or not to be silent on error
 The Superblock Object
 =====================
 A superblock object represents a mounted filesystem.
 struct super_operations
 -----------------------
 This describes how the VFS can manipulate the superblock of your
 filesystem. As of kernel 2.6.13, the following members are defined:
 struct super_operations {
        struct inode *(*alloc_inode)(struct super_block *sb);
        void (*destroy_inode)(struct inode *);
        void (*read_inode) (struct inode *);
        void (*dirty_inode) (struct inode *);
        int (*write_inode) (struct inode *, int);
        void (*put_inode) (struct inode *);
        void (*drop_inode) (struct inode *);
        void (*delete_inode) (struct inode *);
        void (*put_super) (struct super_block *);
        void (*write_super) (struct super_block *);
        int (*sync_fs)(struct super_block *sb, int wait);
        void (*write_super_lockfs) (struct super_block *);
        void (*unlockfs) (struct super_block *);
        int (*statfs) (struct dentry *, struct kstatfs *);
        int (*remount_fs) (struct super_block *, int *, char *);
        void (*clear_inode) (struct inode *);
        void (*umount_begin) (struct super_block *);
        void (*sync_inodes) (struct super_block *sb,
                                struct writeback_control *wbc);
        int (*show_options)(struct seq_file *, struct vfsmount *);
        ssize_t (*quota_read)(struct super_block *, int, char *, size_t, loff_t);
        ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
 };
 All methods are called without any locks being held, unless otherwise
 noted. This means that most methods can block safely. All methods are
 only called from a process context (i.e. not from an interrupt handler
 or bottom half).
  alloc_inode: this method is called by inode_alloc() to allocate memory
 	for struct inode and initialize it.  If this function is not
 	defined, a simple 'struct inode' is allocated.  Normally
 	alloc_inode will be used to allocate a larger structure which
 	contains a 'struct inode' embedded within it.
  destroy_inode: this method is called by destroy_inode() to release
  	resources allocated for struct inode.  It is only required if
  	->alloc_inode was defined and simply undoes anything done by
 	->alloc_inode.
  read_inode: this method is called to read a specific inode from the
        mounted filesystem.  The i_ino member in the struct inode is
 	initialized by the VFS to indicate which inode to read. Other
 	members are filled in by this method.
 	You can set this to NULL and use iget5_locked() instead of iget()
 	to read inodes.  This is necessary for filesystems for which the
 	inode number is not sufficient to identify an inode.
  dirty_inode: this method is called by the VFS to mark an inode dirty.
  write_inode: this method is called when the VFS needs to write an
 	inode to disc.  The second parameter indicates whether the write
 	should be synchronous or not, not all filesystems check this flag.
  put_inode: called when the VFS inode is removed from the inode
 	cache.
  drop_inode: called when the last access to the inode is dropped,
 	with the inode_lock spinlock held.
 	This method should be either NULL (normal UNIX filesystem
 	semantics) or "generic_delete_inode" (for filesystems that do not
 	want to cache inodes - causing "delete_inode" to always be
 	called regardless of the value of i_nlink)
 	The "generic_delete_inode()" behavior is equivalent to the
 	old practice of using "force_delete" in the put_inode() case,
 	but does not have the races that the "force_delete()" approach
 	had. 
  delete_inode: called when the VFS wants to delete an inode
  put_super: called when the VFS wishes to free the superblock
 	(i.e. unmount). This is called with the superblock lock held
  write_super: called when the VFS superblock needs to be written to
 	disc. This method is optional
  sync_fs: called when VFS is writing out all dirty data associated with
  	a superblock. The second parameter indicates whether the method
 	should wait until the write out has been completed. Optional.
  write_super_lockfs: called when VFS is locking a filesystem and
  	forcing it into a consistent state.  This method is currently
  	used by the Logical Volume Manager (LVM).
  unlockfs: called when VFS is unlocking a filesystem and making it writable
  	again.
  statfs: called when the VFS needs to get filesystem statistics. This
 	is called with the kernel lock held
  remount_fs: called when the filesystem is remounted. This is called
 	with the kernel lock held
  clear_inode: called then the VFS clears the inode. Optional
  umount_begin: called when the VFS is unmounting a filesystem.
  sync_inodes: called when the VFS is writing out dirty data associated with
  	a superblock.
  show_options: called by the VFS to show mount options for /proc/<pid>/mounts.
  quota_read: called by the VFS to read from filesystem quota file.
  quota_write: called by the VFS to write to filesystem quota file.
 The read_inode() method is responsible for filling in the "i_op"
 field. This is a pointer to a "struct inode_operations" which
 describes the methods that can be performed on individual inodes.
 The Inode Object
 ================
 An inode object represents an object within the filesystem.
 struct inode_operations
 -----------------------
 This describes how the VFS can manipulate an inode in your
 filesystem. As of kernel 2.6.13, the following members are defined:
 struct inode_operations {
 	int (*create) (struct inode *,struct dentry *,int, struct nameidata *);
 	struct dentry * (*lookup) (struct inode *,struct dentry *, struct nameidata *);
 	int (*link) (struct dentry *,struct inode *,struct dentry *);
 	int (*unlink) (struct inode *,struct dentry *);
 	int (*symlink) (struct inode *,struct dentry *,const char *);
 	int (*mkdir) (struct inode *,struct dentry *,int);
 	int (*rmdir) (struct inode *,struct dentry *);
 	int (*mknod) (struct inode *,struct dentry *,int,dev_t);
 	int (*rename) (struct inode *, struct dentry *,
 			struct inode *, struct dentry *);
 	int (*readlink) (struct dentry *, char __user *,int);
        void * (*follow_link) (struct dentry *, struct nameidata *);
        void (*put_link) (struct dentry *, struct nameidata *, void *);
 	void (*truncate) (struct inode *);
 	int (*permission) (struct inode *, int, struct nameidata *);
 	int (*setattr) (struct dentry *, struct iattr *);
 	int (*getattr) (struct vfsmount *mnt, struct dentry *, struct kstat *);
 	int (*setxattr) (struct dentry *, const char *,const void *,size_t,int);
 	ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t);
 	ssize_t (*listxattr) (struct dentry *, char *, size_t);
 	int (*removexattr) (struct dentry *, const char *);
 };
 Again, all methods are called without any locks being held, unless
 otherwise noted.
  create: called by the open(2) and creat(2) system calls. Only
 	required if you want to support regular files. The dentry you
 	get should not have an inode (i.e. it should be a negative
 	dentry). Here you will probably call d_instantiate() with the
 	dentry and the newly created inode
  lookup: called when the VFS needs to look up an inode in a parent
 	directory. The name to look for is found in the dentry. This
 	method must call d_add() to insert the found inode into the
 	dentry. The "i_count" field in the inode structure should be
 	incremented. If the named inode does not exist a NULL inode
 	should be inserted into the dentry (this is called a negative
 	dentry). Returning an error code from this routine must only
 	be done on a real error, otherwise creating inodes with system
 	calls like create(2), mknod(2), mkdir(2) and so on will fail.
 	If you wish to overload the dentry methods then you should
 	initialise the "d_dop" field in the dentry; this is a pointer
 	to a struct "dentry_operations".
 	This method is called with the directory inode semaphore held
  link: called by the link(2) system call. Only required if you want
 	to support hard links. You will probably need to call
 	d_instantiate() just as you would in the create() method
  unlink: called by the unlink(2) system call. Only required if you
 	want to support deleting inodes
  symlink: called by the symlink(2) system call. Only required if you
 	want to support symlinks. You will probably need to call
 	d_instantiate() just as you would in the create() method
  mkdir: called by the mkdir(2) system call. Only required if you want
 	to support creating subdirectories. You will probably need to
 	call d_instantiate() just as you would in the create() method
  rmdir: called by the rmdir(2) system call. Only required if you want
 	to support deleting subdirectories
  mknod: called by the mknod(2) system call to create a device (char,
 	block) inode or a named pipe (FIFO) or socket. Only required
 	if you want to support creating these types of inodes. You
 	will probably need to call d_instantiate() just as you would
 	in the create() method
  rename: called by the rename(2) system call to rename the object to
 	have the parent and name given by the second inode and dentry.
  readlink: called by the readlink(2) system call. Only required if
 	you want to support reading symbolic links
  follow_link: called by the VFS to follow a symbolic link to the
 	inode it points to.  Only required if you want to support
 	symbolic links.  This method returns a void pointer cookie
 	that is passed to put_link().
  put_link: called by the VFS to release resources allocated by
  	follow_link().  The cookie returned by follow_link() is passed
-  	to to this method as the last parameter.  It is used by
+  	to this method as the last parameter.  It is used by
  	filesystems such as NFS where page cache is not stable
  	(i.e. page that was installed when the symbolic link walk
  	started might not be in the page cache at the end of the
  	walk).
  truncate: called by the VFS to change the size of a file.  The
 	i_size field of the inode is set to the desired size by the
 	VFS before this method is called.  This method is called by
 	the truncate(2) system call and related functionality.
  permission: called by the VFS to check for access rights on a POSIX-like
  	filesystem.
  setattr: called by the VFS to set attributes for a file. This method
  	is called by chmod(2) and related system calls.
  getattr: called by the VFS to get attributes of a file. This method
  	is called by stat(2) and related system calls.
  setxattr: called by the VFS to set an extended attribute for a file.
  	Extended attribute is a name:value pair associated with an
  	inode. This method is called by setxattr(2) system call.
  getxattr: called by the VFS to retrieve the value of an extended
  	attribute name. This method is called by getxattr(2) function
  	call.
  listxattr: called by the VFS to list all extended attributes for a
  	given file. This method is called by listxattr(2) system call.
  removexattr: called by the VFS to remove an extended attribute from
  	a file. This method is called by removexattr(2) system call.
 The Address Space Object
 ========================
 The address space object is used to group and manage pages in the page
 cache.  It can be used to keep track of the pages in a file (or
 anything else) and also track the mapping of sections of the file into
 process address spaces.
 There are a number of distinct yet related services that an
 address-space can provide.  These include communicating memory
 pressure, page lookup by address, and keeping track of pages tagged as
 Dirty or Writeback.
 The first can be used independently to the others.  The VM can try to
 either write dirty pages in order to clean them, or release clean
 pages in order to reuse them.  To do this it can call the ->writepage
 method on dirty pages, and ->releasepage on clean pages with
 PagePrivate set. Clean pages without PagePrivate and with no external
 references will be released without notice being given to the
 address_space.
 To achieve this functionality, pages need to be placed on an LRU with
 lru_cache_add and mark_page_active needs to be called whenever the
 page is used.
 Pages are normally kept in a radix tree index by ->index. This tree
 maintains information about the PG_Dirty and PG_Writeback status of
 each page, so that pages with either of these flags can be found
 quickly.
 The Dirty tag is primarily used by mpage_writepages - the default
 ->writepages method.  It uses the tag to find dirty pages to call
 ->writepage on.  If mpage_writepages is not used (i.e. the address
 provides its own ->writepages) , the PAGECACHE_TAG_DIRTY tag is
 almost unused.  write_inode_now and sync_inode do use it (through
 __sync_single_inode) to check if ->writepages has been successful in
 writing out the whole address_space.
 The Writeback tag is used by filemap*wait* and sync_page* functions,
 via wait_on_page_writeback_range, to wait for all writeback to
 complete.  While waiting ->sync_page (if defined) will be called on
 each page that is found to require writeback.
 An address_space handler may attach extra information to a page,
 typically using the 'private' field in the 'struct page'.  If such
 information is attached, the PG_Private flag should be set.  This will
 cause various VM routines to make extra calls into the address_space
 handler to deal with that data.
 An address space acts as an intermediate between storage and
 application.  Data is read into the address space a whole page at a
 time, and provided to the application either by copying of the page,
 or by memory-mapping the page.
 Data is written into the address space by the application, and then
 written-back to storage typically in whole pages, however the
 address_space has finer control of write sizes.
 The read process essentially only requires 'readpage'.  The write
 process is more complicated and uses prepare_write/commit_write or
 set_page_dirty to write data into the address_space, and writepage,
 sync_page, and writepages to writeback data to storage.
 Adding and removing pages to/from an address_space is protected by the
 inode's i_mutex.
 When data is written to a page, the PG_Dirty flag should be set.  It
 typically remains set until writepage asks for it to be written.  This
 should clear PG_Dirty and set PG_Writeback.  It can be actually
 written at any point after PG_Dirty is clear.  Once it is known to be
 safe, PG_Writeback is cleared.
 Writeback makes use of a writeback_control structure...
 struct address_space_operations
 -------------------------------
 This describes how the VFS can manipulate mapping of a file to page cache in
 your filesystem. As of kernel 2.6.16, the following members are defined:
 struct address_space_operations {
 	int (*writepage)(struct page *page, struct writeback_control *wbc);
 	int (*readpage)(struct file *, struct page *);
 	int (*sync_page)(struct page *);
 	int (*writepages)(struct address_space *, struct writeback_control *);
 	int (*set_page_dirty)(struct page *page);
 	int (*readpages)(struct file *filp, struct address_space *mapping,
 			struct list_head *pages, unsigned nr_pages);
 	int (*prepare_write)(struct file *, struct page *, unsigned, unsigned);
 	int (*commit_write)(struct file *, struct page *, unsigned, unsigned);
 	sector_t (*bmap)(struct address_space *, sector_t);
 	int (*invalidatepage) (struct page *, unsigned long);
 	int (*releasepage) (struct page *, int);
 	ssize_t (*direct_IO)(int, struct kiocb *, const struct iovec *iov,
 			loff_t offset, unsigned long nr_segs);
 	struct page* (*get_xip_page)(struct address_space *, sector_t,
 			int);
 	/* migrate the contents of a page to the specified target */
 	int (*migratepage) (struct page *, struct page *);
 };
  writepage: called by the VM to write a dirty page to backing store.
      This may happen for data integrity reasons (i.e. 'sync'), or
      to free up memory (flush).  The difference can be seen in
      wbc->sync_mode.
      The PG_Dirty flag has been cleared and PageLocked is true.
      writepage should start writeout, should set PG_Writeback,
      and should make sure the page is unlocked, either synchronously
      or asynchronously when the write operation completes.
      If wbc->sync_mode is WB_SYNC_NONE, ->writepage doesn't have to
      try too hard if there are problems, and may choose to write out
      other pages from the mapping if that is easier (e.g. due to
      internal dependencies).  If it chooses not to start writeout, it
      should return AOP_WRITEPAGE_ACTIVATE so that the VM will not keep
      calling ->writepage on that page.
      See the file "Locking" for more details.
  readpage: called by the VM to read a page from backing store.
       The page will be Locked when readpage is called, and should be
       unlocked and marked uptodate once the read completes.
       If ->readpage discovers that it needs to unlock the page for
       some reason, it can do so, and then return AOP_TRUNCATED_PAGE.
       In this case, the page will be relocated, relocked and if
       that all succeeds, ->readpage will be called again.
  sync_page: called by the VM to notify the backing store to perform all
  	queued I/O operations for a page. I/O operations for other pages
 	associated with this address_space object may also be performed.
 	This function is optional and is called only for pages with
  	PG_Writeback set while waiting for the writeback to complete.
  writepages: called by the VM to write out pages associated with the
  	address_space object.  If wbc->sync_mode is WBC_SYNC_ALL, then
  	the writeback_control will specify a range of pages that must be
  	written out.  If it is WBC_SYNC_NONE, then a nr_to_write is given
 	and that many pages should be written if possible.
 	If no ->writepages is given, then mpage_writepages is used
  	instead.  This will choose pages from the address space that are
  	tagged as DIRTY and will pass them to ->writepage.
  set_page_dirty: called by the VM to set a page dirty.
        This is particularly needed if an address space attaches
        private data to a page, and that data needs to be updated when
        a page is dirtied.  This is called, for example, when a memory
 	mapped page gets modified.
 	If defined, it should set the PageDirty flag, and the
        PAGECACHE_TAG_DIRTY tag in the radix tree.
  readpages: called by the VM to read pages associated with the address_space
  	object. This is essentially just a vector version of
  	readpage.  Instead of just one page, several pages are
  	requested.
 	readpages is only used for read-ahead, so read errors are
  	ignored.  If anything goes wrong, feel free to give up.
  prepare_write: called by the generic write path in VM to set up a write
  	request for a page.  This indicates to the address space that
  	the given range of bytes is about to be written.  The
  	address_space should check that the write will be able to
  	complete, by allocating space if necessary and doing any other
  	internal housekeeping.  If the write will update parts of
  	any basic-blocks on storage, then those blocks should be
  	pre-read (if they haven't been read already) so that the
  	updated blocks can be written out properly.
 	The page will be locked.  If prepare_write wants to unlock the
  	page it, like readpage, may do so and return
  	AOP_TRUNCATED_PAGE.
 	In this case the prepare_write will be retried one the lock is
  	regained.
  commit_write: If prepare_write succeeds, new data will be copied
        into the page and then commit_write will be called.  It will
        typically update the size of the file (if appropriate) and
        mark the inode as dirty, and do any other related housekeeping
        operations.  It should avoid returning an error if possible -
        errors should have been handled by prepare_write.
  bmap: called by the VFS to map a logical block offset within object to
  	physical block number. This method is used by the FIBMAP
  	ioctl and for working with swap-files.  To be able to swap to
  	a file, the file must have a stable mapping to a block
  	device.  The swap system does not go through the filesystem
  	but instead uses bmap to find out where the blocks in the file
  	are and uses those addresses directly.
  invalidatepage: If a page has PagePrivate set, then invalidatepage
        will be called when part or all of the page is to be removed
 	from the address space.  This generally corresponds to either a
 	truncation or a complete invalidation of the address space
 	(in the latter case 'offset' will always be 0).
 	Any private data associated with the page should be updated
 	to reflect this truncation.  If offset is 0, then
 	the private data should be released, because the page
 	must be able to be completely discarded.  This may be done by
        calling the ->releasepage function, but in this case the
        release MUST succeed.
  releasepage: releasepage is called on PagePrivate pages to indicate
        that the page should be freed if possible.  ->releasepage
        should remove any private data from the page and clear the
        PagePrivate flag.  It may also remove the page from the
        address_space.  If this fails for some reason, it may indicate
        failure with a 0 return value.
 	This is used in two distinct though related cases.  The first
        is when the VM finds a clean page with no active users and
        wants to make it a free page.  If ->releasepage succeeds, the
        page will be removed from the address_space and become free.
 	The second case if when a request has been made to invalidate
        some or all pages in an address_space.  This can happen
        through the fadvice(POSIX_FADV_DONTNEED) system call or by the
        filesystem explicitly requesting it as nfs and 9fs do (when
        they believe the cache may be out of date with storage) by
        calling invalidate_inode_pages2().
 	If the filesystem makes such a call, and needs to be certain
        that all pages are invalidated, then its releasepage will
        need to ensure this.  Possibly it can clear the PageUptodate
        bit if it cannot free private data yet.
  direct_IO: called by the generic read/write routines to perform
        direct_IO - that is IO requests which bypass the page cache
        and transfer data directly between the storage and the
        application's address space.
  get_xip_page: called by the VM to translate a block number to a page.
 	The page is valid until the corresponding filesystem is unmounted.
 	Filesystems that want to use execute-in-place (XIP) need to implement
 	it.  An example implementation can be found in fs/ext2/xip.c.
  migrate_page:  This is used to compact the physical memory usage.
        If the VM wants to relocate a page (maybe off a memory card
        that is signalling imminent failure) it will pass a new page
 	and an old page to this function.  migrate_page should
 	transfer any private data across and update any references
        that it has to the page.
 The File Object
 ===============
 A file object represents a file opened by a process.
 struct file_operations
 ----------------------
 This describes how the VFS can manipulate an open file. As of kernel
 2.6.17, the following members are defined:
 struct file_operations {
 	loff_t (*llseek) (struct file *, loff_t, int);
 	ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
 	ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
 	ssize_t (*aio_read) (struct kiocb *, const struct iovec *, unsigned long, loff_t);
 	ssize_t (*aio_write) (struct kiocb *, const struct iovec *, unsigned long, loff_t);
 	int (*readdir) (struct file *, void *, filldir_t);
 	unsigned int (*poll) (struct file *, struct poll_table_struct *);
 	int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
 	long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
 	long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
 	int (*mmap) (struct file *, struct vm_area_struct *);
 	int (*open) (struct inode *, struct file *);
 	int (*flush) (struct file *);
 	int (*release) (struct inode *, struct file *);
 	int (*fsync) (struct file *, struct dentry *, int datasync);
 	int (*aio_fsync) (struct kiocb *, int datasync);
 	int (*fasync) (int, struct file *, int);
 	int (*lock) (struct file *, int, struct file_lock *);
 	ssize_t (*readv) (struct file *, const struct iovec *, unsigned long, loff_t *);
 	ssize_t (*writev) (struct file *, const struct iovec *, unsigned long, loff_t *);
 	ssize_t (*sendfile) (struct file *, loff_t *, size_t, read_actor_t, void *);
 	ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int);
 	unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
 	int (*check_flags)(int);
 	int (*dir_notify)(struct file *filp, unsigned long arg);
 	int (*flock) (struct file *, int, struct file_lock *);
 	ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, size_t, unsigned 
 int);
 	ssize_t (*splice_read)(struct file *, struct pipe_inode_info *, size_t, unsigned  
 int);
 };
 Again, all methods are called without any locks being held, unless
 otherwise noted.
  llseek: called when the VFS needs to move the file position index
  read: called by read(2) and related system calls
  aio_read: called by io_submit(2) and other asynchronous I/O operations
  write: called by write(2) and related system calls
  aio_write: called by io_submit(2) and other asynchronous I/O operations
  readdir: called when the VFS needs to read the directory contents
  poll: called by the VFS when a process wants to check if there is
 	activity on this file and (optionally) go to sleep until there
 	is activity. Called by the select(2) and poll(2) system calls
  ioctl: called by the ioctl(2) system call
  unlocked_ioctl: called by the ioctl(2) system call. Filesystems that do not
  	require the BKL should use this method instead of the ioctl() above.
  compat_ioctl: called by the ioctl(2) system call when 32 bit system calls
 	 are used on 64 bit kernels.
  mmap: called by the mmap(2) system call
  open: called by the VFS when an inode should be opened. When the VFS
 	opens a file, it creates a new "struct file". It then calls the
 	open method for the newly allocated file structure. You might
 	think that the open method really belongs in
 	"struct inode_operations", and you may be right. I think it's
 	done the way it is because it makes filesystems simpler to
 	implement. The open() method is a good place to initialize the
 	"private_data" member in the file structure if you want to point
 	to a device structure
  flush: called by the close(2) system call to flush a file
  release: called when the last reference to an open file is closed
  fsync: called by the fsync(2) system call
  fasync: called by the fcntl(2) system call when asynchronous
 	(non-blocking) mode is enabled for a file
  lock: called by the fcntl(2) system call for F_GETLK, F_SETLK, and F_SETLKW
  	commands
  readv: called by the readv(2) system call
  writev: called by the writev(2) system call
  sendfile: called by the sendfile(2) system call
  get_unmapped_area: called by the mmap(2) system call
  check_flags: called by the fcntl(2) system call for F_SETFL command
  dir_notify: called by the fcntl(2) system call for F_NOTIFY command
  flock: called by the flock(2) system call
  splice_write: called by the VFS to splice data from a pipe to a file. This
 		method is used by the splice(2) system call
  splice_read: called by the VFS to splice data from file to a pipe. This
 	       method is used by the splice(2) system call
 Note that the file operations are implemented by the specific
 filesystem in which the inode resides. When opening a device node
 (character or block special) most filesystems will call special
 support routines in the VFS which will locate the required device
 driver information. These support routines replace the filesystem file
 operations with those for the device driver, and then proceed to call
 the new open() method for the file. This is how opening a device file
 in the filesystem eventually ends up calling the device driver open()
 method.
 Directory Entry Cache (dcache)
 ==============================
 struct dentry_operations
 ------------------------
 This describes how a filesystem can overload the standard dentry
 operations. Dentries and the dcache are the domain of the VFS and the
 individual filesystem implementations. Device drivers have no business
 here. These methods may be set to NULL, as they are either optional or
 the VFS uses a default. As of kernel 2.6.13, the following members are
 defined:
 struct dentry_operations {
 	int (*d_revalidate)(struct dentry *, struct nameidata *);
 	int (*d_hash) (struct dentry *, struct qstr *);
 	int (*d_compare) (struct dentry *, struct qstr *, struct qstr *);
 	int (*d_delete)(struct dentry *);
 	void (*d_release)(struct dentry *);
 	void (*d_iput)(struct dentry *, struct inode *);
 };
  d_revalidate: called when the VFS needs to revalidate a dentry. This
 	is called whenever a name look-up finds a dentry in the
 	dcache. Most filesystems leave this as NULL, because all their
 	dentries in the dcache are valid
  d_hash: called when the VFS adds a dentry to the hash table
  d_compare: called when a dentry should be compared with another
  d_delete: called when the last reference to a dentry is
 	deleted. This means no-one is using the dentry, however it is
 	still valid and in the dcache
  d_release: called when a dentry is really deallocated
  d_iput: called when a dentry loses its inode (just prior to its
 	being deallocated). The default when this is NULL is that the
 	VFS calls iput(). If you define this method, you must call
 	iput() yourself
 Each dentry has a pointer to its parent dentry, as well as a hash list
 of child dentries. Child dentries are basically like files in a
 directory.
 Directory Entry Cache API
 --------------------------
 There are a number of functions defined which permit a filesystem to
 manipulate dentries:
  dget: open a new handle for an existing dentry (this just increments
 	the usage count)
  dput: close a handle for a dentry (decrements the usage count). If
 	the usage count drops to 0, the "d_delete" method is called
 	and the dentry is placed on the unused list if the dentry is
 	still in its parents hash list. Putting the dentry on the
 	unused list just means that if the system needs some RAM, it
 	goes through the unused list of dentries and deallocates them.
 	If the dentry has already been unhashed and the usage count
 	drops to 0, in this case the dentry is deallocated after the
 	"d_delete" method is called
  d_drop: this unhashes a dentry from its parents hash list. A
 	subsequent call to dput() will deallocate the dentry if its
 	usage count drops to 0
  d_delete: delete a dentry. If there are no other open references to
 	the dentry then the dentry is turned into a negative dentry
 	(the d_iput() method is called). If there are other
 	references, then d_drop() is called instead
  d_add: add a dentry to its parents hash list and then calls
 	d_instantiate()
  d_instantiate: add a dentry to the alias hash list for the inode and
 	updates the "d_inode" member. The "i_count" member in the
 	inode structure should be set/incremented. If the inode
 	pointer is NULL, the dentry is called a "negative
 	dentry". This function is commonly called when an inode is
 	created for an existing negative dentry
  d_lookup: look up a dentry given its parent and path name component
 	It looks up the child of that given name from the dcache
 	hash table. If it is found, the reference count is incremented
 	and the dentry is returned. The caller must use d_put()
 	to free the dentry when it finishes using it.
 For further information on dentry locking, please refer to the document
 Documentation/filesystems/dentry-locking.txt.
 Resources
 =========
 (Note some of these resources are not up-to-date with the latest kernel
 version.)
 Creating Linux virtual filesystems. 2002
    <http://lwn.net/Articles/13325/>
 The Linux Virtual File-system Layer by Neil Brown. 1999
    <http://www.cse.unsw.edu.au/~neilb/oss/linux-commentary/vfs.html>
 A tour of the Linux VFS by Michael K. Johnson. 1996
    <http://www.tldp.org/LDP/khg/HyperNews/get/fs/vfstour.html>
 A small trail through the Linux kernel by Andries Brouwer. 2001
    <http://www.win.tue.nl/~aeb/linux/vfs/trail.html>
 				 =================================
 				 FR451 MMU LINUX MEMORY MANAGEMENT
 				 =================================
 ============
 MMU HARDWARE
 ============
 FR451 MMU Linux puts the MMU into EDAT mode whilst running. This means that it uses both the SAT
 registers and the DAT TLB to perform address translation.
 There are 8 IAMLR/IAMPR register pairs and 16 DAMLR/DAMPR register pairs for SAT mode.
 In DAT mode, there is also a TLB organised in cache format as 64 lines x 2 ways. Each line spans a
 16KB range of addresses, but can match a larger region.
 ===========================
 MEMORY MANAGEMENT REGISTERS
 ===========================
 Certain control registers are used by the kernel memory management routines:
 	REGISTERS		USAGE
 	======================	==================================================
 	IAMR0, DAMR0		Kernel image and data mappings
 	IAMR1, DAMR1		First-chance TLB lookup mapping
 	DAMR2			Page attachment for cache flush by page
 	DAMR3			Current PGD mapping
 	SCR0, DAMR4		Instruction TLB PGE/PTD cache
 	SCR1, DAMR5		Data TLB PGE/PTD cache
 	DAMR6-10		kmap_atomic() mappings
 	DAMR11			I/O mapping
 	CXNR			mm_struct context ID
 	TTBR			Page directory (PGD) pointer (physical address)
 =====================
 GENERAL MEMORY LAYOUT
 =====================
 The physical memory layout is as follows:
  PHYSICAL ADDRESS	CONTROLLER	DEVICE
  ===================	==============	=======================================
  00000000 - BFFFFFFF	SDRAM		SDRAM area
  E0000000 - EFFFFFFF	L-BUS CS2#	VDK SLBUS/PCI window
  F0000000 - F0FFFFFF	L-BUS CS5#	MB93493 CSC area (DAV daughter board)
  F1000000 - F1FFFFFF	L-BUS CS7#	(CB70 CPU-card PCMCIA port I/O space)
  FC000000 - FC0FFFFF	L-BUS CS1#	VDK MB86943 config space
  FC100000 - FC1FFFFF	L-BUS CS6#	DM9000 NIC I/O space
  FC200000 - FC2FFFFF	L-BUS CS3#	MB93493 CSR area (DAV daughter board)
  FD000000 - FDFFFFFF	L-BUS CS4#	(CB70 CPU-card extra flash space)
  FE000000 - FEFFFFFF			Internal CPU peripherals
  FF000000 - FF1FFFFF	L-BUS CS0#	Flash 1
  FF200000 - FF3FFFFF	L-BUS CS0#	Flash 2
  FFC00000 - FFC0001F	L-BUS CS0#	FPGA
 The virtual memory layout is:
  VIRTUAL ADDRESS    PHYSICAL	TRANSLATOR	FLAGS	SIZE	OCCUPATION
  =================  ========	==============	=======	=======	===================================
  00004000-BFFFFFFF  various	TLB,xAMR1	D-N-??V	3GB	Userspace
  C0000000-CFFFFFFF  00000000	xAMPR0		-L-S--V	256MB	Kernel image and data
  D0000000-D7FFFFFF  various	TLB,xAMR1	D-NS??V	128MB	vmalloc area
  D8000000-DBFFFFFF  various	TLB,xAMR1	D-NS??V	64MB	kmap() area
  DC000000-DCFFFFFF  various	TLB			1MB	Secondary kmap_atomic() frame
  DD000000-DD27FFFF  various	DAMR			160KB	Primary kmap_atomic() frame
  DD040000			DAMR2/IAMR2	-L-S--V	page	Page cache flush attachment point
  DD080000			DAMR3		-L-SC-V	page	Page Directory (PGD)
  DD0C0000			DAMR4		-L-SC-V	page	Cached insn TLB Page Table lookup
  DD100000			DAMR5		-L-SC-V	page	Cached data TLB Page Table lookup
  DD140000			DAMR6		-L-S--V	page	kmap_atomic(KM_BOUNCE_READ)
  DD180000			DAMR7		-L-S--V	page	kmap_atomic(KM_SKB_SUNRPC_DATA)
  DD1C0000			DAMR8		-L-S--V	page	kmap_atomic(KM_SKB_DATA_SOFTIRQ)
  DD200000			DAMR9		-L-S--V	page	kmap_atomic(KM_USER0)
  DD240000			DAMR10		-L-S--V	page	kmap_atomic(KM_USER1)
  E0000000-FFFFFFFF  E0000000	DAMR11		-L-SC-V	512MB	I/O region
 IAMPR1 and DAMPR1 are used as an extension to the TLB.
 ====================
 KMAP AND KMAP_ATOMIC
 ====================
 To access pages in the page cache (which may not be directly accessible if highmem is available),
 the kernel calls kmap(), does the access and then calls kunmap(); or it calls kmap_atomic(), does
 the access and then calls kunmap_atomic().
 kmap() creates an attachment between an arbitrary inaccessible page and a range of virtual
 addresses by installing a PTE in a special page table. The kernel can then access this page as it
 wills. When it's finished, the kernel calls kunmap() to clear the PTE.
 kmap_atomic() does something slightly different. In the interests of speed, it chooses one of two
 strategies:
 (1) If possible, kmap_atomic() attaches the requested page to one of DAMPR5 through DAMPR10
     register pairs; and the matching kunmap_atomic() clears the DAMPR. This makes high memory
     support really fast as there's no need to flush the TLB or modify the page tables. The DAMLR
     registers being used for this are preset during boot and don't change over the lifetime of the
     process. There's a direct mapping between the first few kmap_atomic() types, DAMR number and
     virtual address slot.
     However, there are more kmap_atomic() types defined than there are DAMR registers available,
     so we fall back to:
 (2) kmap_atomic() uses a slot in the secondary frame (determined by the type parameter), and then
     locks an entry in the TLB to translate that slot to the specified page. The number of slots is
     obviously limited, and their positions are controlled such that each slot is matched by a
     different line in the TLB. kunmap() ejects the entry from the TLB.
 Note that the first three kmap atomic types are really just declared as placeholders. The DAMPR
 registers involved are actually modified directly.
 Also note that kmap() itself may sleep, kmap_atomic() may never sleep and both always succeed;
 furthermore, a driver using kmap() may sleep before calling kunmap(), but may not sleep before
 calling kunmap_atomic() if it had previously called kmap_atomic().
 ===============================
 USING MORE THAN 256MB OF MEMORY
 ===============================
 The kernel cannot access more than 256MB of memory directly. The physical layout, however, permits
 up to 3GB of SDRAM (possibly 3.25GB) to be made available. By using CONFIG_HIGHMEM, the kernel can
 allow userspace (by way of page tables) and itself (by way of kmap) to deal with the memory
 allocation.
 External devices can, of course, still DMA to and from all of the SDRAM, even if the kernel can't
 see it directly. The kernel translates page references into real addresses for communicating to the
 devices.
 ===================
 PAGE TABLE TOPOLOGY
 ===================
 The page tables are arranged in 2-layer format. There is a middle layer (PMD) that would be used in
 3-layer format tables but that is folded into the top layer (PGD) and so consumes no extra memory
 or processing power.
  +------+     PGD    PMD
  | TTBR |--->+-------------------+
  +------+    |      |      : STE |
              | PGE0 | PME0 : STE |
              |      |      : STE |
              +-------------------+              Page Table
              |      |      : STE -------------->+--------+ +0x0000
              | PGE1 | PME0 : STE -----------+   | PTE0   |
              |      |      : STE -------+   |   +--------+
              +-------------------+      |   |   | PTE63  |
              |      |      : STE |      |   +-->+--------+ +0x0100
              | PGE2 | PME0 : STE |      |       | PTE64  |
              |      |      : STE |      |       +--------+
              +-------------------+      |       | PTE127 |
              |      |      : STE |      +------>+--------+ +0x0200
              | PGE3 | PME0 : STE |              | PTE128 |
              |      |      : STE |              +--------+
              +-------------------+              | PTE191 |
                                                 +--------+ +0x0300
 Each Page Directory (PGD) is 16KB (page size) in size and is divided into 64 entries (PGEs). Each
 PGE contains one Page Mid Directory (PMD).
 Each PMD is 256 bytes in size and contains a single entry (PME). Each PME holds 64 FR451 MMU
 segment table entries of 4 bytes apiece. Each PME "points to" a page table. In practice, each STE
 points to a subset of the page table, the first to PT+0x0000, the second to PT+0x0100, the third to
 PT+0x200, and so on.
 Each PGE and PME covers 64MB of the total virtual address space.
 Each Page Table (PTD) is 16KB (page size) in size, and is divided into 4096 entries (PTEs). Each
 entry can point to one 16KB page. In practice, each Linux page table is subdivided into 64 FR451
 MMU page tables. But they are all grouped together to make management easier, in particular rmap
 support is then trivial.
 Grouping page tables in this fashion makes PGE caching in SCR0/SCR1 more efficient because the
 coverage of the cached item is greater.
 Page tables for the vmalloc area are allocated at boot time and shared between all mm_structs.
 =================
 USER SPACE LAYOUT
 =================
 For MMU capable Linux, the regions userspace code are allowed to access are kept entirely separate
 from those dedicated to the kernel:
 	VIRTUAL ADDRESS    SIZE   PURPOSE
 	=================  =====  ===================================
 	00000000-00003fff  4KB    NULL pointer access trap
 	00004000-01ffffff  ~32MB  lower mmap space (grows up)
 	02000000-021fffff  2MB    Stack space (grows down from top)
 	02200000-nnnnnnnn         Executable mapping
        nnnnnnnn-                 brk space (grows up)
 	        -bfffffff         upper mmap space (grows down)
 This is so arranged so as to make best use of the 16KB page tables and the way in which PGEs/PMEs
 are cached by the TLB handler. The lower mmap space is filled first, and then the upper mmap space
 is filled.
 ===============================
 GDB-STUB MMU DEBUGGING SERVICES
 ===============================
 The gdb-stub included in this kernel provides a number of services to aid in the debugging of MMU
 related kernel services:
 (*) Every time the kernel stops, certain state information is dumped into __debug_mmu. This
     variable is defined in arch/frv/kernel/gdb-stub.c. Note that the gdbinit file in this
     directory has some useful macros for dealing with this.
     (*) __debug_mmu.tlb[]
 	 This receives the current TLB contents. This can be viewed with the _tlb GDB macro:
 		(gdb) _tlb
 		tlb[0x00]: 01000005 00718203  01000002 00718203
 		tlb[0x01]: 01004002 006d4201  01004005 006d4203
 		tlb[0x02]: 01008002 006d0201  01008006 00004200
 		tlb[0x03]: 0100c006 007f4202  0100c002 0064c202
 		tlb[0x04]: 01110005 00774201  01110002 00774201
 		tlb[0x05]: 01114005 00770201  01114002 00770201
 		tlb[0x06]: 01118002 0076c201  01118005 0076c201
 		...
 		tlb[0x3d]: 010f4002 00790200  001f4002 0054ca02
 		tlb[0x3e]: 010f8005 0078c201  010f8002 0078c201
 		tlb[0x3f]: 001fc002 0056ca01  001fc005 00538a01
     (*) __debug_mmu.iamr[]
     (*) __debug_mmu.damr[]
-	 These receive the current IAMR and DAMR contents. These can be viewed with with the _amr
+	 These receive the current IAMR and DAMR contents. These can be viewed with the _amr
 	 GDB macro:
 		(gdb) _amr
 		AMRx           DAMR                    IAMR
 		====   =====================   =====================
 		amr0 : L:c0000000 P:00000cb9 : L:c0000000 P:000004b9
 		amr1 : L:01070005 P:006f9203 : L:0102c005 P:006a1201
 		amr2 : L:d8d00000 P:00000000 : L:d8d00000 P:00000000
 		amr3 : L:d8d04000 P:00534c0d : L:00000000 P:00000000
 		amr4 : L:d8d08000 P:00554c0d : L:00000000 P:00000000
 		amr5 : L:d8d0c000 P:00554c0d : L:00000000 P:00000000
 		amr6 : L:d8d10000 P:00000000 : L:00000000 P:00000000
 		amr7 : L:d8d14000 P:00000000 : L:00000000 P:00000000
 		amr8 : L:d8d18000 P:00000000
 		amr9 : L:d8d1c000 P:00000000
 		amr10: L:d8d20000 P:00000000
 		amr11: L:e0000000 P:e0000ccd
 (*) The current task's page directory is bound to DAMR3.
     This can be viewed with the _pgd GDB macro:
 	(gdb) _pgd
 	$3 = {{pge = {{ste = {0x554001, 0x554101, 0x554201, 0x554301, 0x554401,
 		  0x554501, 0x554601, 0x554701, 0x554801, 0x554901, 0x554a01,
 		  0x554b01, 0x554c01, 0x554d01, 0x554e01, 0x554f01, 0x555001,
 		  0x555101, 0x555201, 0x555301, 0x555401, 0x555501, 0x555601,
 		  0x555701, 0x555801, 0x555901, 0x555a01, 0x555b01, 0x555c01,
 		  0x555d01, 0x555e01, 0x555f01, 0x556001, 0x556101, 0x556201,
 		  0x556301, 0x556401, 0x556501, 0x556601, 0x556701, 0x556801,
 		  0x556901, 0x556a01, 0x556b01, 0x556c01, 0x556d01, 0x556e01,
 		  0x556f01, 0x557001, 0x557101, 0x557201, 0x557301, 0x557401,
 		  0x557501, 0x557601, 0x557701, 0x557801, 0x557901, 0x557a01,
 		  0x557b01, 0x557c01, 0x557d01, 0x557e01, 0x557f01}}}}, {pge = {{
 		ste = {0x0 <repeats 64 times>}}}} <repeats 51 times>, {pge = {{ste = {
 		  0x248001, 0x248101, 0x248201, 0x248301, 0x248401, 0x248501,
 		  0x248601, 0x248701, 0x248801, 0x248901, 0x248a01, 0x248b01,
 		  0x248c01, 0x248d01, 0x248e01, 0x248f01, 0x249001, 0x249101,
 		  0x249201, 0x249301, 0x249401, 0x249501, 0x249601, 0x249701,
 		  0x249801, 0x249901, 0x249a01, 0x249b01, 0x249c01, 0x249d01,
 		  0x249e01, 0x249f01, 0x24a001, 0x24a101, 0x24a201, 0x24a301,
 		  0x24a401, 0x24a501, 0x24a601, 0x24a701, 0x24a801, 0x24a901,
 		  0x24aa01, 0x24ab01, 0x24ac01, 0x24ad01, 0x24ae01, 0x24af01,
 		  0x24b001, 0x24b101, 0x24b201, 0x24b301, 0x24b401, 0x24b501,
 		  0x24b601, 0x24b701, 0x24b801, 0x24b901, 0x24ba01, 0x24bb01,
 		  0x24bc01, 0x24bd01, 0x24be01, 0x24bf01}}}}, {pge = {{ste = {
 		  0x0 <repeats 64 times>}}}} <repeats 11 times>}
 (*) The PTD last used by the instruction TLB miss handler is attached to DAMR4.
 (*) The PTD last used by the data TLB miss handler is attached to DAMR5.
     These can be viewed with the _ptd_i and _ptd_d GDB macros:
 	(gdb) _ptd_d
 	$5 = {{pte = 0x0} <repeats 127 times>, {pte = 0x539b01}, {
 	    pte = 0x0} <repeats 896 times>, {pte = 0x719303}, {pte = 0x6d5303}, {
 	    pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {
 	    pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x6a1303}, {
 	    pte = 0x0} <repeats 12 times>, {pte = 0x709303}, {pte = 0x0}, {pte = 0x0},
 	  {pte = 0x6fd303}, {pte = 0x6f9303}, {pte = 0x6f5303}, {pte = 0x0}, {
 	    pte = 0x6ed303}, {pte = 0x531b01}, {pte = 0x50db01}, {
 	    pte = 0x0} <repeats 13 times>, {pte = 0x5303}, {pte = 0x7f5303}, {
 	    pte = 0x509b01}, {pte = 0x505b01}, {pte = 0x7c9303}, {pte = 0x7b9303}, {
 	    pte = 0x7b5303}, {pte = 0x7b1303}, {pte = 0x7ad303}, {pte = 0x0}, {
 	    pte = 0x0}, {pte = 0x7a1303}, {pte = 0x0}, {pte = 0x795303}, {pte = 0x0}, {
 	    pte = 0x78d303}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {pte = 0x0}, {
 	    pte = 0x0}, {pte = 0x775303}, {pte = 0x771303}, {pte = 0x76d303}, {
 	    pte = 0x0}, {pte = 0x765303}, {pte = 0x7c5303}, {pte = 0x501b01}, {
 	    pte = 0x4f1b01}, {pte = 0x4edb01}, {pte = 0x0}, {pte = 0x4f9b01}, {
 	    pte = 0x4fdb01}, {pte = 0x0} <repeats 2992 times>}
 EFI Real Time Clock driver
 -------------------------------
 S. Eranian <eranian@hpl.hp.com>
 March 2000
 I/ Introduction
 This document describes the efirtc.c driver has provided for
 the IA-64 platform. 
 The purpose of this driver is to supply an API for kernel and user applications
 to get access to the Time Service offered by EFI version 0.92.
 EFI provides 4 calls one can make once the OS is booted: GetTime(),
 SetTime(), GetWakeupTime(), SetWakeupTime() which are all supported by this
 driver. We describe those calls as well the design of the driver in the
 following sections.
 II/ Design Decisions
 The original ideas was to provide a very simple driver to get access to, 
 at first, the time of day service. This is required in order to access, in a 
 portable way, the CMOS clock. A program like /sbin/hwclock uses such a clock 
 to initialize the system view of the time during boot.
 Because we wanted to minimize the impact on existing user-level apps using
 the CMOS clock, we decided to expose an API that was very similar to the one
 used today with the legacy RTC driver (driver/char/rtc.c). However, because 
-EFI provides a simpler services, not all all ioctl() are available. Also
+EFI provides a simpler services, not all ioctl() are available. Also
 new ioctl()s have been introduced for things that EFI provides but not the 
 legacy.
 EFI uses a slightly different way of representing the time, noticeably
 the reference date is different. Year is the using the full 4-digit format.
 The Epoch is January 1st 1998. For backward compatibility reasons we don't
 expose this new way of representing time. Instead we use something very 
 similar to the struct tm, i.e. struct rtc_time, as used by hwclock.
 One of the reasons for doing it this way is to allow for EFI to still evolve
 without necessarily impacting any of the user applications. The decoupling
 enables flexibility and permits writing wrapper code is ncase things change.
 The driver exposes two interfaces, one via the device file and a set of
 ioctl()s. The other is read-only via the /proc filesystem. 
 As of today we don't offer a /proc/sys interface.
 To allow for a uniform interface between the legacy RTC and EFI time service,
 we have created the include/linux/rtc.h header file to contain only the 
 "public" API of the two drivers.  The specifics of the legacy RTC are still 
 in include/linux/mc146818rtc.h.
 III/ Time of day service
 The part of the driver gives access to the time of day service of EFI.
 Two ioctl()s, compatible with the legacy RTC calls:
 	Read the CMOS clock: ioctl(d, RTC_RD_TIME, &rtc);
 	Write the CMOS clock: ioctl(d, RTC_SET_TIME, &rtc);
 The rtc is a pointer to a data structure defined in rtc.h which is close
 to a struct tm:
 struct rtc_time {
        int tm_sec;
        int tm_min;
        int tm_hour;
        int tm_mday;
        int tm_mon;
        int tm_year;
        int tm_wday;
        int tm_yday;
        int tm_isdst;
 };
 The driver takes care of converting back an forth between the EFI time and
 this format.
 Those two ioctl()s can be exercised with the hwclock command:
 For reading:
 # /sbin/hwclock --show
 Mon Mar  6 15:32:32 2000  -0.910248 seconds
 For setting:
 # /sbin/hwclock --systohc
 Root privileges are required to be able to set the time of day.
 IV/ Wakeup Alarm service
 EFI provides an API by which one can program when a machine should wakeup,
 i.e. reboot. This is very different from the alarm provided by the legacy
 RTC which is some kind of interval timer alarm. For this reason we don't use
 the same ioctl()s to get access to the service. Instead we have
 introduced 2 news ioctl()s to the interface of an RTC. 
 We have added 2 new ioctl()s that are specific to the EFI driver:
 	Read the current state of the alarm
 	ioctl(d, RTC_WKLAM_RD, &wkt)
 	Set the alarm or change its status
 	ioctl(d, RTC_WKALM_SET, &wkt)
 The wkt structure encapsulates a struct rtc_time + 2 extra fields to get 
 status information:
 struct rtc_wkalrm {
        unsigned char enabled; /* =1 if alarm is enabled */
        unsigned char pending; /* =1 if alarm is pending  */
        struct rtc_time time;
 } 
 As of today, none of the existing user-level apps supports this feature.
 However writing such a program should be hard by simply using those two 
 ioctl(). 
 Root privileges are required to be able to set the alarm.
 V/ References.
 Checkout the following Web site for more information on EFI:
 http://developer.intel.com/technology/efi/
 An ad-hoc collection of notes on IA64 MCA and INIT processing.  Feel
 free to update it with notes about any area that is not clear.
 ---
 MCA/INIT are completely asynchronous.  They can occur at any time, when
 the OS is in any state.  Including when one of the cpus is already
 holding a spinlock.  Trying to get any lock from MCA/INIT state is
 asking for deadlock.  Also the state of structures that are protected
 by locks is indeterminate, including linked lists.
 ---
 The complicated ia64 MCA process.  All of this is mandated by Intel's
-specification for ia64 SAL, error recovery and and unwind, it is not as
+specification for ia64 SAL, error recovery and unwind, it is not as
 if we have a choice here.
 * MCA occurs on one cpu, usually due to a double bit memory error.
  This is the monarch cpu.
 * SAL sends an MCA rendezvous interrupt (which is a normal interrupt)
  to all the other cpus, the slaves.
 * Slave cpus that receive the MCA interrupt call down into SAL, they
  end up spinning disabled while the MCA is being serviced.
 * If any slave cpu was already spinning disabled when the MCA occurred
  then it cannot service the MCA interrupt.  SAL waits ~20 seconds then
  sends an unmaskable INIT event to the slave cpus that have not
  already rendezvoused.
 * Because MCA/INIT can be delivered at any time, including when the cpu
  is down in PAL in physical mode, the registers at the time of the
  event are _completely_ undefined.  In particular the MCA/INIT
  handlers cannot rely on the thread pointer, PAL physical mode can
  (and does) modify TP.  It is allowed to do that as long as it resets
  TP on return.  However MCA/INIT events expose us to these PAL
  internal TP changes.  Hence curr_task().
 * If an MCA/INIT event occurs while the kernel was running (not user
  space) and the kernel has called PAL then the MCA/INIT handler cannot
  assume that the kernel stack is in a fit state to be used.  Mainly
  because PAL may or may not maintain the stack pointer internally.
  Because the MCA/INIT handlers cannot trust the kernel stack, they
  have to use their own, per-cpu stacks.  The MCA/INIT stacks are
  preformatted with just enough task state to let the relevant handlers
  do their job.
 * Unlike most other architectures, the ia64 struct task is embedded in
  the kernel stack[1].  So switching to a new kernel stack means that
  we switch to a new task as well.  Because various bits of the kernel
  assume that current points into the struct task, switching to a new
  stack also means a new value for current.
 * Once all slaves have rendezvoused and are spinning disabled, the
  monarch is entered.  The monarch now tries to diagnose the problem
  and decide if it can recover or not.
 * Part of the monarch's job is to look at the state of all the other
  tasks.  The only way to do that on ia64 is to call the unwinder,
  as mandated by Intel.
 * The starting point for the unwind depends on whether a task is
  running or not.  That is, whether it is on a cpu or is blocked.  The
  monarch has to determine whether or not a task is on a cpu before it
  knows how to start unwinding it.  The tasks that received an MCA or
  INIT event are no longer running, they have been converted to blocked
  tasks.  But (and its a big but), the cpus that received the MCA
  rendezvous interrupt are still running on their normal kernel stacks!
 * To distinguish between these two cases, the monarch must know which
  tasks are on a cpu and which are not.  Hence each slave cpu that
  switches to an MCA/INIT stack, registers its new stack using
  set_curr_task(), so the monarch can tell that the _original_ task is
  no longer running on that cpu.  That gives us a decent chance of
  getting a valid backtrace of the _original_ task.
 * MCA/INIT can be nested, to a depth of 2 on any cpu.  In the case of a
  nested error, we want diagnostics on the MCA/INIT handler that
  failed, not on the task that was originally running.  Again this
  requires set_curr_task() so the MCA/INIT handlers can register their
  own stack as running on that cpu.  Then a recursive error gets a
  trace of the failing handler's "task".
 [1] My (Keith Owens) original design called for ia64 to separate its
    struct task and the kernel stacks.  Then the MCA/INIT data would be
    chained stacks like i386 interrupt stacks.  But that required
    radical surgery on the rest of ia64, plus extra hard wired TLB
    entries with its associated performance degradation.  David
    Mosberger vetoed that approach.  Which meant that separate kernel
    stacks meant separate "tasks" for the MCA/INIT handlers.
 ---
 INIT is less complicated than MCA.  Pressing the nmi button or using
 the equivalent command on the management console sends INIT to all
-cpus.  SAL picks one one of the cpus as the monarch and the rest are
+cpus.  SAL picks one of the cpus as the monarch and the rest are
 slaves.  All the OS INIT handlers are entered at approximately the same
 time.  The OS monarch prints the state of all tasks and returns, after
 which the slaves return and the system resumes.
 At least that is what is supposed to happen.  Alas there are broken
 versions of SAL out there.  Some drive all the cpus as monarchs.  Some
 drive them all as slaves.  Some drive one cpu as monarch, wait for that
 cpu to return from the OS then drive the rest as slaves.  Some versions
 of SAL cannot even cope with returning from the OS, they spin inside
 SAL on resume.  The OS INIT code has workarounds for some of these
 broken SAL symptoms, but some simply cannot be fixed from the OS side.
 ---
 The scheduler hooks used by ia64 (curr_task, set_curr_task) are layer
 violations.  Unfortunately MCA/INIT start off as massive layer
 violations (can occur at _any_ time) and they build from there.
 At least ia64 makes an attempt at recovering from hardware errors, but
 it is a difficult problem because of the asynchronous nature of these
 errors.  When processing an unmaskable interrupt we sometimes need
 special code to cope with our inability to take any locks.
 ---
 How is ia64 MCA/INIT different from x86 NMI?
 * x86 NMI typically gets delivered to one cpu.  MCA/INIT gets sent to
  all cpus.
 * x86 NMI cannot be nested.  MCA/INIT can be nested, to a depth of 2
  per cpu.
 * x86 has a separate struct task which points to one of multiple kernel
  stacks.  ia64 has the struct task embedded in the single kernel
  stack, so switching stack means switching task.
 * x86 does not call the BIOS so the NMI handler does not have to worry
  about any registers having changed.  MCA/INIT can occur while the cpu
  is in PAL in physical mode, with undefined registers and an undefined
  kernel stack.
 * i386 backtrace is not very sensitive to whether a process is running
  or not.  ia64 unwind is very, very sensitive to whether a process is
  running or not.
 ---
 What happens when MCA/INIT is delivered what a cpu is running user
 space code?
 The user mode registers are stored in the RSE area of the MCA/INIT on
 entry to the OS and are restored from there on return to SAL, so user
 mode registers are preserved across a recoverable MCA/INIT.  Since the
 OS has no idea what unwind data is available for the user space stack,
 MCA/INIT never tries to backtrace user space.  Which means that the OS
 does not bother making the user space process look like a blocked task,
 i.e. the OS does not copy pt_regs and switch_stack to the user space
 stack.  Also the OS has no idea how big the user space RSE and memory
 stacks are, which makes it too risky to copy the saved state to a user
 mode stack.
 ---
 How do we get a backtrace on the tasks that were running when MCA/INIT
 was delivered?
 mca.c:::ia64_mca_modify_original_stack().  That identifies and
 verifies the original kernel stack, copies the dirty registers from
 the MCA/INIT stack's RSE to the original stack's RSE, copies the
 skeleton struct pt_regs and switch_stack to the original stack, fills
 in the skeleton structures from the PAL minstate area and updates the
 original stack's thread.ksp.  That makes the original stack look
 exactly like any other blocked task, i.e. it now appears to be
 sleeping.  To get a backtrace, just start with thread.ksp for the
 original task and unwind like any other sleeping task.
 ---
 How do we identify the tasks that were running when MCA/INIT was
 delivered?
 If the previous task has been verified and converted to a blocked
 state, then sos->prev_task on the MCA/INIT stack is updated to point to
 the previous task.  You can look at that field in dumps or debuggers.
 To help distinguish between the handler and the original tasks,
 handlers have _TIF_MCA_INIT set in thread_info.flags.
 The sos data is always in the MCA/INIT handler stack, at offset
 MCA_SOS_OFFSET.  You can get that value from mca_asm.h or calculate it
 as KERNEL_STACK_SIZE - sizeof(struct pt_regs) - sizeof(struct
 ia64_sal_os_state), with 16 byte alignment for all structures.
 Also the comm field of the MCA/INIT task is modified to include the pid
 of the original task, for humans to use.  For example, a comm field of
 'MCA 12159' means that pid 12159 was running when the MCA was
 delivered.
 			  Linux Input drivers v1.0
 	       (c) 1999-2001 Vojtech Pavlik <vojtech@ucw.cz>
 			     Sponsored by SuSE
 	    $Id: input.txt,v 1.8 2002/05/29 03:15:01 bradleym Exp $
 ----------------------------------------------------------------------------
 0. Disclaimer
 ~~~~~~~~~~~~~
  This program is free software; you can redistribute it and/or modify it
 under the terms of the GNU General Public License as published by the Free
 Software Foundation; either version 2 of the License, or (at your option)
 any later version.
  This program is distributed in the hope that it will be useful, but
 WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
 or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 more details.
  You should have received a copy of the GNU General Public License along
 with this program; if not, write to the Free Software Foundation, Inc., 59
 Temple Place, Suite 330, Boston, MA 02111-1307 USA
  Should you need to contact me, the author, you can do so either by e-mail
 - mail your message to <vojtech@ucw.cz>, or by paper mail: Vojtech Pavlik,
 Simunkova 1594, Prague 8, 182 00 Czech Republic
  For your convenience, the GNU General Public License version 2 is included
 in the package: See the file COPYING.
 1. Introduction
 ~~~~~~~~~~~~~~~
  This is a collection of drivers that is designed to support all input
 devices under Linux. While it is currently used only on for USB input
 devices, future use (say 2.5/2.6) is expected to expand to replace
 most of the existing input system, which is why it lives in
 drivers/input/ instead of drivers/usb/.
  The centre of the input drivers is the input module, which must be
 loaded before any other of the input modules - it serves as a way of
 communication between two groups of modules:
 1.1 Device drivers
 ~~~~~~~~~~~~~~~~~~
  These modules talk to the hardware (for example via USB), and provide
 events (keystrokes, mouse movements) to the input module.
 1.2 Event handlers
 ~~~~~~~~~~~~~~~~~~
  These modules get events from input and pass them where needed via
 various interfaces - keystrokes to the kernel, mouse movements via a
 simulated PS/2 interface to GPM and X and so on.
 2. Simple Usage
 ~~~~~~~~~~~~~~~
  For the most usual configuration, with one USB mouse and one USB keyboard,
 you'll have to load the following modules (or have them built in to the
 kernel):
 	input
 	mousedev
 	keybdev
 	usbcore
 	uhci_hcd or ohci_hcd or ehci_hcd
 	usbhid
  After this, the USB keyboard will work straight away, and the USB mouse
 will be available as a character device on major 13, minor 63:
 	crw-r--r--   1 root     root      13,  63 Mar 28 22:45 mice
  This device has to be created.
  The commands to create it by hand are:
 	cd /dev
 	mkdir input
 	mknod input/mice c 13 63
  After that you have to point GPM (the textmode mouse cut&paste tool) and
 XFree to this device to use it - GPM should be called like:
 	gpm -t ps2 -m /dev/input/mice
  And in X:
 	Section "Pointer"
 	    Protocol    "ImPS/2"
 	    Device      "/dev/input/mice"
 	    ZAxisMapping 4 5
 	EndSection
  When you do all of the above, you can use your USB mouse and keyboard.
 3. Detailed Description
 ~~~~~~~~~~~~~~~~~~~~~~~
 3.1 Device drivers
 ~~~~~~~~~~~~~~~~~~
  Device drivers are the modules that generate events. The events are
 however not useful without being handled, so you also will need to use some
 of the modules from section 3.2.
 3.1.1 usbhid
 ~~~~~~~~~~~~
  usbhid is the largest and most complex driver of the whole suite. It
 handles all HID devices, and because there is a very wide variety of them,
 and because the USB HID specification isn't simple, it needs to be this big.
  Currently, it handles USB mice, joysticks, gamepads, steering wheels
 keyboards, trackballs and digitizers.
 However, USB uses HID also for monitor controls, speaker controls, UPSs,
 LCDs and many other purposes.
 The monitor and speaker controls should be easy to add to the hid/input
 interface, but for the UPSs and LCDs it doesn't make much sense. For this,
 the hiddev interface was designed. See Documentation/usb/hiddev.txt
 for more information about it.
  The usage of the usbhid module is very simple, it takes no parameters,
 detects everything automatically and when a HID device is inserted, it
 detects it appropriately.
  However, because the devices vary wildly, you might happen to have a
 device that doesn't work well. In that case #define DEBUG at the beginning
 of hid-core.c and send me the syslog traces.
 3.1.2 usbmouse
 ~~~~~~~~~~~~~~
  For embedded systems, for mice with broken HID descriptors and just any
 other use when the big usbhid wouldn't be a good choice, there is the
 usbmouse driver. It handles USB mice only. It uses a simpler HIDBP
 protocol. This also means the mice must support this simpler protocol. Not
 all do. If you don't have any strong reason to use this module, use usbhid
 instead.
 3.1.3 usbkbd
 ~~~~~~~~~~~~
  Much like usbmouse, this module talks to keyboards with a simplified
 HIDBP protocol. It's smaller, but doesn't support any extra special keys.
 Use usbhid instead if there isn't any special reason to use this.
 3.1.4 wacom
 ~~~~~~~~~~~
  This is a driver for Wacom Graphire and Intuos tablets. Not for Wacom
 PenPartner, that one is handled by the HID driver. Although the Intuos and
 Graphire tablets claim that they are HID tablets as well, they are not and
 thus need this specific driver.
 3.1.5 iforce
 ~~~~~~~~~~~~
  A driver for I-Force joysticks and wheels, both over USB and RS232. 
 It includes ForceFeedback support now, even though Immersion
 Corp. considers the protocol a trade secret and won't disclose a word
 about it. 
 3.2 Event handlers
 ~~~~~~~~~~~~~~~~~~
  Event handlers distribute the events from the devices to userland and
 kernel, as needed.
 3.2.1 keybdev
 ~~~~~~~~~~~~~
  keybdev is currently a rather ugly hack that translates the input
 events into architecture-specific keyboard raw mode (Xlated AT Set2 on
 x86), and passes them into the handle_scancode function of the
 keyboard.c module. This works well enough on all architectures that
 keybdev can generate rawmode on, other architectures can be added to
 it.
  The right way would be to pass the events to keyboard.c directly,
 best if keyboard.c would itself be an event handler. This is done in
 the input patch, available on the webpage mentioned below. 
 3.2.2 mousedev
 ~~~~~~~~~~~~~~
  mousedev is also a hack to make programs that use mouse input
 work. It takes events from either mice or digitizers/tablets and makes
 a PS/2-style (a la /dev/psaux) mouse device available to the
 userland. Ideally, the programs could use a more reasonable interface,
 for example evdev
  Mousedev devices in /dev/input (as shown above) are:
 	crw-r--r--   1 root     root      13,  32 Mar 28 22:45 mouse0
 	crw-r--r--   1 root     root      13,  33 Mar 29 00:41 mouse1
 	crw-r--r--   1 root     root      13,  34 Mar 29 00:41 mouse2
 	crw-r--r--   1 root     root      13,  35 Apr  1 10:50 mouse3
 	...
 	...
 	crw-r--r--   1 root     root      13,  62 Apr  1 10:50 mouse30
 	crw-r--r--   1 root     root      13,  63 Apr  1 10:50 mice
 Each 'mouse' device is assigned to a single mouse or digitizer, except
 the last one - 'mice'. This single character device is shared by all
 mice and digitizers, and even if none are connected, the device is
 present.  This is useful for hotplugging USB mice, so that programs
 can open the device even when no mice are present.
  CONFIG_INPUT_MOUSEDEV_SCREEN_[XY] in the kernel configuration are
 the size of your screen (in pixels) in XFree86. This is needed if you
 want to use your digitizer in X, because its movement is sent to X
 via a virtual PS/2 mouse and thus needs to be scaled
 accordingly. These values won't be used if you use a mouse only.
  Mousedev will generate either PS/2, ImPS/2 (Microsoft IntelliMouse) or
 ExplorerPS/2 (IntelliMouse Explorer) protocols, depending on what the
 program reading the data wishes. You can set GPM and X to any of
 these. You'll need ImPS/2 if you want to make use of a wheel on a USB
 mouse and ExplorerPS/2 if you want to use extra (up to 5) buttons. 
 3.2.3 joydev
 ~~~~~~~~~~~~
  Joydev implements v0.x and v1.x Linux joystick api, much like
 drivers/char/joystick/joystick.c used to in earlier versions. See
 joystick-api.txt in the Documentation subdirectory for details.  As
 soon as any joystick is connected, it can be accessed in /dev/input
 on: 
 	crw-r--r--   1 root     root      13,   0 Apr  1 10:50 js0
 	crw-r--r--   1 root     root      13,   1 Apr  1 10:50 js1
 	crw-r--r--   1 root     root      13,   2 Apr  1 10:50 js2
 	crw-r--r--   1 root     root      13,   3 Apr  1 10:50 js3
 	...
 And so on up to js31.
 3.2.4 evdev
 ~~~~~~~~~~~
  evdev is the generic input event interface. It passes the events
 generated in the kernel straight to the program, with timestamps. The
 API is still evolving, but should be useable now. It's described in
 section 5. 
-  This should be the way for GPM and X to get keyboard and mouse mouse
+  This should be the way for GPM and X to get keyboard and mouse
 events. It allows for multihead in X without any specific multihead
 kernel support. The event codes are the same on all architectures and
 are hardware independent.
  The devices are in /dev/input:
 	crw-r--r--   1 root     root      13,  64 Apr  1 10:49 event0
 	crw-r--r--   1 root     root      13,  65 Apr  1 10:50 event1
 	crw-r--r--   1 root     root      13,  66 Apr  1 10:50 event2
 	crw-r--r--   1 root     root      13,  67 Apr  1 10:50 event3
 	...
 And so on up to event31.
 4. Verifying if it works
 ~~~~~~~~~~~~~~~~~~~~~~~~
  Typing a couple keys on the keyboard should be enough to check that
 a USB keyboard works and is correctly connected to the kernel keyboard
 driver. 
  Doing a cat /dev/input/mouse0 (c, 13, 32) will verify that a mouse
 is also emulated, characters should appear if you move it.
  You can test the joystick emulation with the 'jstest' utility,
 available in the joystick package (see Documentation/input/joystick.txt).
  You can test the event devices with the 'evtest' utility available
 in the LinuxConsole project CVS archive (see the URL below).
 5. Event interface
 ~~~~~~~~~~~~~~~~~~
  Should you want to add event device support into any application (X, gpm,
 svgalib ...) I <vojtech@ucw.cz> will be happy to provide you any help I
 can. Here goes a description of the current state of things, which is going
 to be extended, but not changed incompatibly as time goes:
  You can use blocking and nonblocking reads, also select() on the
 /dev/input/eventX devices, and you'll always get a whole number of input
 events on a read. Their layout is:
 struct input_event {
 	struct timeval time;
 	unsigned short type;
 	unsigned short code;
 	unsigned int value;
 };
  'time' is the timestamp, it returns the time at which the event happened.
 Type is for example EV_REL for relative moment, REL_KEY for a keypress or
 release. More types are defined in include/linux/input.h.
  'code' is event code, for example REL_X or KEY_BACKSPACE, again a complete
 list is in include/linux/input.h.
  'value' is the value the event carries. Either a relative change for
 EV_REL, absolute new value for EV_ABS (joysticks ...), or 0 for EV_KEY for
 release, 1 for keypress and 2 for autorepeat.
 $Id: INTERFACE.fax,v 1.2 2000/08/06 09:22:50 armin Exp $
 Description of the fax-subinterface between linklevel and hardwarelevel of 
  isdn4linux. 
  The communication between linklevel (LL) and hardwarelevel (HL) for fax
  is based on the struct T30_s (defined in isdnif.h).
  This struct is allocated in the LL.  
  In order to use fax, the LL provides the pointer to this struct with the 
  command ISDN_CMD_SETL3 (parm.fax). This pointer expires in case of hangup 
  and when a new channel to a new connection is assigned. 
 Data handling:
  In send-mode the HL-driver has to handle the <DLE> codes and the bit-order 
  conversion by itself. 
  In receive-mode the LL-driver takes care of the bit-order conversion
  (specified by +FBOR)
 Structure T30_s description:
  This structure stores the values (set by AT-commands), the remote-
  capability-values and the command-codes between LL and HL.
  If the HL-driver receives ISDN_CMD_FAXCMD, all needed information
  is in this struct set by the LL.
  To signal information to the LL, the HL-driver has to set the 
-  the parameters and use ISDN_STAT_FAXIND.
+  parameters and use ISDN_STAT_FAXIND.
  (Please refer to INTERFACE)
 Structure T30_s:
  All members are 8-bit unsigned (__u8)
  -  resolution     
  -  rate
  -  width
  -  length
  -  compression
  -  ecm
  -  binary
  -  scantime
  -  id[]
  Local faxmachine's parameters, set by +FDIS, +FDCS, +FLID, ...
  -  r_resolution
  -  r_rate
  -  r_width
  -  r_length
  -  r_compression
  -  r_ecm
  -  r_binary
  -  r_scantime
  -  r_id[]
  Remote faxmachine's parameters. To be set by HL-driver.
  -  phase      
  Defines the actual state of fax connection. Set by HL or LL
  depending on progress and type of connection.
  If the phase changes because of an AT command, the LL driver
  changes this value. Otherwise the HL-driver takes care of it, but
  only necessary on call establishment (from IDLE to PHASE_A).
  (one of the constants ISDN_FAX_PHASE_[IDLE,A,B,C,D,E])
  -  direction
  Defines outgoing/send or incoming/receive connection.
  (ISDN_TTY_FAX_CONN_[IN,OUT])
  -  code
  Commands from LL to HL; possible constants : 
      ISDN_TTY_FAX_DR        signals +FDR command to HL
      ISDN_TTY_FAX_DT        signals +FDT command to HL 
      ISDN_TTY_FAX_ET        signals +FET command to HL
  Other than that the "code" is set with the hangup-code value at
  the end of connection for the +FHNG message.
  -  r_code 
  Commands from HL to LL; possible constants :
      ISDN_TTY_FAX_CFR       output of +FCFR message. 
      ISDN_TTY_FAX_RID       output of remote ID set in r_id[]
                             (+FCSI/+FTSI on send/receive)
      ISDN_TTY_FAX_DCS       output of +FDCS and CONNECT message,
                             switching to phase C.
      ISDN_TTY_FAX_ET        signals end of data,
                             switching to phase D.
      ISDN_TTY_FAX_FCON      signals the established, outgoing connection,
                             switching to phase B.
      ISDN_TTY_FAX_FCON_I    signals the established, incoming connection,
                             switching to phase B.
      ISDN_TTY_FAX_DIS       output of +FDIS message and values.
      ISDN_TTY_FAX_SENT      signals that all data has been sent 
                             and <DLE><ETX> is acknowledged,
                             OK message will be sent.
      ISDN_TTY_FAX_PTS       signals a msg-confirmation (page sent successful),
                             depending on fet value:
                             0: output OK message (more pages follow)
                             1: switching to phase B (next document)
      ISDN_TTY_FAX_TRAIN_OK  output of +FDCS and OK message (for receive mode).
      ISDN_TTY_FAX_EOP       signals end of data in receive mode,
                             switching to phase D.
      ISDN_TTY_FAX_HNG       output of the +FHNG and value set by code and
                             OK message, switching to phase E.
  -  badlin
  Value of +FBADLIN  
  -  badmul
  Value of +FBADMUL
  -  bor
  Value of +FBOR
  -  fet
  Value of +FET command in send-mode.
  Set by HL in receive-mode for +FET message.
  -  pollid[]  
  ID-string, set by +FCIG
  -  cq
  Value of +FCQ
  -  cr
  Value of +FCR
  -  ctcrty
  Value of +FCTCRTY
  -  minsp
  Value of +FMINSP
  -  phcto
  Value of +FPHCTO
  -  rel
  Value of +FREL
  -  nbc
  Value of +FNBC (0,1)
  (+FNBC is not a known class 2 fax command, I added this to change the
   automatic "best capabilities" connection in the eicon HL-driver)
 Armin
 mac@melware.de
 $Id: README.hysdn,v 1.3.6.1 2001/02/10 14:41:19 kai Exp $
 The hysdn driver has been written by
-by Werner Cornelius (werner@isdn4linux.de or werner@titro.de) 
+Werner Cornelius (werner@isdn4linux.de or werner@titro.de)
 for Hypercope GmbH Aachen Germany. Hypercope agreed to publish this driver
 under the GNU General Public License.
 The CAPI 2.0-support was added by Ulrich Albrecht (ualbrecht@hypercope.de)
 for Hypercope GmbH Aachen, Germany.
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 Table of contents
 =================
 1. About the driver
 2. Loading/Unloading the driver
 3. Entries in the /proc filesystem
 4. The /proc/net/hysdn/cardconfX file
 5. The /proc/net/hysdn/cardlogX file
 6. Where to get additional info and help
 1. About the driver
   The drivers/isdn/hysdn subdir contains a driver for HYPERCOPEs active 
   PCI isdn cards Champ, Ergo and Metro. To enable support for this cards
   enable ISDN support in the kernel config and support for HYSDN cards in
   the active cards submenu. The driver may only be compiled and used if 
   support for loadable modules and the process filesystem have been enabled.
   These cards provide two different interfaces to the kernel. Without the
   optional CAPI 2.0 support, they register as ethernet card. IP-routing
   to a ISDN-destination is performed on the card itself. All necessary
   handlers for various protocols like ppp and others as well as config info
   and firmware may be fetched from Hypercopes WWW-Site www.hypercope.de.
   With CAPI 2.0 support enabled, the card can also be used as a CAPI 2.0 
   compliant devices with either CAPI 2.0 applications 
   (check isdn4k-utils) or -using the capidrv module- as a regular
   isdn4linux device. This is done via the same mechanism as with the 
   active AVM cards and in fact uses the same module.
 2. Loading/Unloading the driver
   The module has no command line parameters and auto detects up to 10 cards
   in the id-range 0-9.
   If a loaded driver shall be unloaded all open files in the /proc/net/hysdn
   subdir need to be closed and all ethernet interfaces allocated by this 
   driver must be shut down. Otherwise the module counter will avoid a module
   unload.
   If you are using the CAPI 2.0-interface, make sure to load/modprobe the
   kernelcapi-module first.
   If you plan to use the capidrv-link to isdn4linux, make sure to load
   capidrv.o after all modules using this driver (i.e. after hysdn and
   any avm-specific modules).
 3. Entries in the /proc filesystem
   When the module has been loaded it adds the directory hysdn in the 
   /proc/net tree. This directory contains exactly 2 file entries for each 
   card. One is called cardconfX and the other cardlogX, where X is the
   card id number from 0 to 9. 
   The cards are numbered in the order found in the PCI config data.
 4. The /proc/net/hysdn/cardconfX file
   This file may be read to get by everyone to get info about the cards type, 
   actual state, available features and used resources.
   The first 3 entries (id, bus and slot) are PCI info fields, the following
   type field gives the information about the cards type:
   4 -> Ergo card (server card with 2 b-chans)
   5 -> Metro card (server card with 4 or 8 b-chans)
   6 -> Champ card (client card with 2 b-chans)   
   The following 3 fields show the hardware assignments for irq, iobase and the
   dual ported memory (dp-mem).
   The fields b-chans and fax-chans announce the available card resources of
   this types for the user.
   The state variable indicates the actual drivers state for this card with the
   following assignments.
   0 -> card has not been booted since driver load
   1 -> card booting is actually in progess
   2 -> card is in an error state due to a previous boot failure
   3 -> card is booted and active
   And the last field (device) shows the name of the ethernet device assigned
   to this card. Up to the first successful boot this field only shows a -
   to tell that no net device has been allocated up to now. Once a net device
   has been allocated it remains assigned to this card, even if a card is
   rebooted and an boot error occurs. 
   Writing to the cardconfX file boots the card or transfers config lines to 
   the cards firmware. The type of data is automatically detected when the 
   first data is written. Only root has write access to this file.
   The firmware boot files are normally called hyclient.pof for client cards
   and hyserver.pof for server cards.
   After successfully writing the boot file, complete config files or single
   config lines may be copied to this file.
   If an error occurs the return value given to the writing process has the 
   following additional codes (decimal):
   1000 Another process is currently bootng the card
   1001 Invalid firmware header
   1002 Boards dual-port RAM test failed
   1003 Internal firmware handler error
   1004 Boot image size invalid
   1005 First boot stage (bootstrap loader) failed
   1006 Second boot stage failure
   1007 Timeout waiting for card ready during boot
   1008 Operation only allowed in booted state
   1009 Config line too long 
   1010 Invalid channel number 
   1011 Timeout sending config data
   Additional info about error reasons may be fetched from the log output. 
 5. The /proc/net/hysdn/cardlogX file
   The cardlogX file entry may be opened multiple for reading by everyone to 
   get the cards and drivers log data. Card messages always start with the
   keyword LOG. All other lines are output from the driver. 
   The driver log data may be redirected to the syslog by selecting the 
   appropriate bitmask. The cards log messages will always be send to this
   interface but never to the syslog.
   A root user may write a decimal or hex (with 0x) value t this file to select
   desired output options. As mentioned above the cards log dat is always 
   written to the cardlog file independent of the following options only used
   to check and debug the driver itself:
   For example: 
   echo "0x34560078" > /proc/net/hysdn/cardlog0
   to output the hex log mask 34560078 for card 0.
   The written value is regarded as an unsigned 32-Bit value, bit ored for 
   desired output. The following bits are already assigned:
   0x80000000   All driver log data is alternatively via syslog 
   0x00000001   Log memory allocation errors
   0x00000010   Firmware load start and close are logged
   0x00000020   Log firmware record parser
   0x00000040   Log every firmware write actions
   0x00000080   Log all card related boot messages
   0x00000100   Output all config data sent for debugging purposes
   0x00000200   Only non comment config lines are shown wth channel
   0x00000400   Additional conf log output
   0x00001000   Log the asynchronous scheduler actions (config and log)
   0x00100000   Log all open and close actions to /proc/net/hysdn/card files
   0x00200000   Log all actions from /proc file entries
   0x00010000   Log network interface init and deinit
 6. Where to get additional info and help
   If you have any problems concerning the driver or configuration contact 
   the Hypercope support team (support@hypercope.de) and or the authors
   Werner Cornelius (werner@isdn4linux or cornelius@titro.de) or
   Ulrich Albrecht (ualbrecht@hypercope.de).
 ================================================================
 Documentation for Kdump - The kexec-based Crash Dumping Solution
 ================================================================
 This document includes overview, setup and installation, and analysis
 information.
 Overview
 ========
 Kdump uses kexec to quickly boot to a dump-capture kernel whenever a
 dump of the system kernel's memory needs to be taken (for example, when
 the system panics). The system kernel's memory image is preserved across
 the reboot and is accessible to the dump-capture kernel.
 You can use common Linux commands, such as cp and scp, to copy the
 memory image to a dump file on the local disk, or across the network to
 a remote system.
 Kdump and kexec are currently supported on the x86, x86_64, and ppc64
 architectures.
 When the system kernel boots, it reserves a small section of memory for
 the dump-capture kernel. This ensures that ongoing Direct Memory Access
 (DMA) from the system kernel does not corrupt the dump-capture kernel.
 The kexec -p command loads the dump-capture kernel into this reserved
 memory.
 On x86 machines, the first 640 KB of physical memory is needed to boot,
 regardless of where the kernel loads. Therefore, kexec backs up this
 region just before rebooting into the dump-capture kernel.
 All of the necessary information about the system kernel's core image is
 encoded in the ELF format, and stored in a reserved area of memory
 before a crash. The physical address of the start of the ELF header is
 passed to the dump-capture kernel through the elfcorehdr= boot
 parameter.
 With the dump-capture kernel, you can access the memory image, or "old
 memory," in two ways:
 - Through a /dev/oldmem device interface. A capture utility can read the
  device file and write out the memory in raw format. This is a raw dump
  of memory. Analysis and capture tools must be intelligent enough to
  determine where to look for the right information.
 - Through /proc/vmcore. This exports the dump as an ELF-format file that
  you can write out using file copy commands such as cp or scp. Further,
  you can use analysis tools such as the GNU Debugger (GDB) and the Crash
  tool to debug the dump file. This method ensures that the dump pages are
  correctly ordered.
 Setup and Installation
 ======================
 Install kexec-tools and the Kdump patch
 ---------------------------------------
 1) Login as the root user.
 2) Download the kexec-tools user-space package from the following URL:
   http://www.xmission.com/~ebiederm/files/kexec/kexec-tools-1.101.tar.gz
 3) Unpack the tarball with the tar command, as follows:
   tar xvpzf kexec-tools-1.101.tar.gz
 4) Download the latest consolidated Kdump patch from the following URL:
   http://lse.sourceforge.net/kdump/
   (This location is being used until all the user-space Kdump patches
   are integrated with the kexec-tools package.)
 5) Change to the kexec-tools-1.101 directory, as follows:
   cd kexec-tools-1.101
 6) Apply the consolidated patch to the kexec-tools-1.101 source tree
   with the patch command, as follows. (Modify the path to the downloaded
   patch as necessary.)
   patch -p1 < /path-to-kdump-patch/kexec-tools-1.101-kdump.patch
 7) Configure the package, as follows:
   ./configure
 8) Compile the package, as follows:
   make
 9) Install the package, as follows:
   make install
 Download and build the system and dump-capture kernels
 ------------------------------------------------------
 Download the mainline (vanilla) kernel source code (2.6.13-rc1 or newer)
 from http://www.kernel.org. Two kernels must be built: a system kernel
 and a dump-capture kernel. Use the following steps to configure these
 kernels with the necessary kexec and Kdump features:
 System kernel
 -------------
 1) Enable "kexec system call" in "Processor type and features."
   CONFIG_KEXEC=y
 2) Enable "sysfs file system support" in "Filesystem" -> "Pseudo
   filesystems." This is usually enabled by default.
   CONFIG_SYSFS=y
   Note that "sysfs file system support" might not appear in the "Pseudo
   filesystems" menu if "Configure standard kernel features (for small
   systems)" is not enabled in "General Setup." In this case, check the
   .config file itself to ensure that sysfs is turned on, as follows:
   grep 'CONFIG_SYSFS' .config
 3) Enable "Compile the kernel with debug info" in "Kernel hacking."
   CONFIG_DEBUG_INFO=Y
   This causes the kernel to be built with debug symbols. The dump
   analysis tools require a vmlinux with debug symbols in order to read
   and analyze a dump file.
 4) Make and install the kernel and its modules. Update the boot loader
   (such as grub, yaboot, or lilo) configuration files as necessary.
 5) Boot the system kernel with the boot parameter "crashkernel=Y@X",
   where Y specifies how much memory to reserve for the dump-capture kernel
   and X specifies the beginning of this reserved memory. For example,
   "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory
   starting at physical address 0x01000000 for the dump-capture kernel.
   On x86 and x86_64, use "crashkernel=64M@16M".
   On ppc64, use "crashkernel=128M@32M".
 The dump-capture kernel
 -----------------------
 1) Under "General setup," append "-kdump" to the current string in
   "Local version."
 2) On x86, enable high memory support under "Processor type and
   features":
   CONFIG_HIGHMEM64G=y
   or
   CONFIG_HIGHMEM4G
 3) On x86 and x86_64, disable symmetric multi-processing support
   under "Processor type and features":
   CONFIG_SMP=n
   (If CONFIG_SMP=y, then specify maxcpus=1 on the kernel command line
   when loading the dump-capture kernel, see section "Load the Dump-capture
   Kernel".)
 4) On ppc64, disable NUMA support and enable EMBEDDED support:
   CONFIG_NUMA=n
   CONFIG_EMBEDDED=y
   CONFIG_EEH=N for the dump-capture kernel
 5) Enable "kernel crash dumps" support under "Processor type and
   features":
   CONFIG_CRASH_DUMP=y
 6) Use a suitable value for "Physical address where the kernel is
   loaded" (under "Processor type and features"). This only appears when
   "kernel crash dumps" is enabled. By default this value is 0x1000000
   (16MB). It should be the same as X in the "crashkernel=Y@X" boot
   parameter discussed above.
   On x86 and x86_64, use "CONFIG_PHYSICAL_START=0x1000000".
   On ppc64 the value is automatically set at 32MB when
   CONFIG_CRASH_DUMP is set.
 6) Optionally enable "/proc/vmcore support" under "Filesystems" ->
   "Pseudo filesystems".
   CONFIG_PROC_VMCORE=y
   (CONFIG_PROC_VMCORE is set by default when CONFIG_CRASH_DUMP is selected.)
 7) Make and install the kernel and its modules. DO NOT add this kernel
   to the boot loader configuration files.
 Load the Dump-capture Kernel
 ============================
 After booting to the system kernel, load the dump-capture kernel using
 the following command:
   kexec -p <dump-capture-kernel> \
   --initrd=<initrd-for-dump-capture-kernel> --args-linux \
   --append="root=<root-dev> init 1 irqpoll"
 Notes on loading the dump-capture kernel:
 * <dump-capture-kernel> must be a vmlinux image (that is, an
  uncompressed ELF image). bzImage does not work at this time.
 * By default, the ELF headers are stored in ELF64 format to support
  systems with more than 4GB memory. The --elf32-core-headers option can
  be used to force the generation of ELF32 headers. This is necessary
  because GDB currently cannot open vmcore files with ELF64 headers on
  32-bit systems. ELF32 headers can be used on non-PAE systems (that is,
  less than 4GB of memory).
 * The "irqpoll" boot parameter reduces driver initialization failures
  due to shared interrupts in the dump-capture kernel.
 * You must specify <root-dev> in the format corresponding to the root
  device name in the output of mount command.
 * "init 1" boots the dump-capture kernel into single-user mode without
  networking. If you want networking, use "init 3."
 Kernel Panic
 ============
 After successfully loading the dump-capture kernel as previously
 described, the system will reboot into the dump-capture kernel if a
 system crash is triggered.  Trigger points are located in panic(),
 die(), die_nmi() and in the sysrq handler (ALT-SysRq-c).
 The following conditions will execute a crash trigger point:
 If a hard lockup is detected and "NMI watchdog" is configured, the system
 will boot into the dump-capture kernel ( die_nmi() ).
 If die() is called, and it happens to be a thread with pid 0 or 1, or die()
 is called inside interrupt context or die() is called and panic_on_oops is set,
 the system will boot into the dump-capture kernel.
-On powererpc systems when a soft-reset is generated, die() is called by all cpus and the system system will boot into the dump-capture kernel.
+On powererpc systems when a soft-reset is generated, die() is called by all cpus and the system will boot into the dump-capture kernel.
 For testing purposes, you can trigger a crash by using "ALT-SysRq-c",
 "echo c > /proc/sysrq-trigger or write a module to force the panic.
 Write Out the Dump File
 =======================
 After the dump-capture kernel is booted, write out the dump file with
 the following command:
   cp /proc/vmcore <dump-file>
 You can also access dumped memory as a /dev/oldmem device for a linear
 and raw view. To create the device, use the following command:
    mknod /dev/oldmem c 1 12
 Use the dd command with suitable options for count, bs, and skip to
 access specific portions of the dump.
 To see the entire memory, use the following command:
   dd if=/dev/oldmem of=oldmem.001
 Analysis
 ========
 Before analyzing the dump image, you should reboot into a stable kernel.
 You can do limited analysis using GDB on the dump file copied out of
 /proc/vmcore. Use the debug vmlinux built with -g and run the following
 command:
   gdb vmlinux <dump-file>
 Stack trace for the task on processor 0, register display, and memory
 display work fine.
 Note: GDB cannot analyze core files generated in ELF64 format for x86.
 On systems with a maximum of 4GB of memory, you can generate
 ELF32-format headers using the --elf32-core-headers kernel option on the
 dump kernel.
 You can also use the Crash utility to analyze dump files in Kdump
 format. Crash is available on Dave Anderson's site at the following URL:
   http://people.redhat.com/~anderson/
 To Do
 =====
 1) Provide a kernel pages filtering mechanism, so core file size is not
   extreme on systems with huge memory banks.
 2) Relocatable kernel can help in maintaining multiple kernels for
   crash_dump, and the same kernel as the system kernel can be used to
   capture the dump.
 Contact
 =======
 Vivek Goyal (vgoyal@in.ibm.com)
 Maneesh Soni (maneesh@in.ibm.com)
 Trademark
 =========
 Linux is a trademark of Linus Torvalds in the United States, other
 countries, or both.
 			 ============================
 			 KERNEL KEY RETENTION SERVICE
 			 ============================
 This service allows cryptographic keys, authentication tokens, cross-domain
 user mappings, and similar to be cached in the kernel for the use of
 filesystems other kernel services.
 Keyrings are permitted; these are a special type of key that can hold links to
 other keys. Processes each have three standard keyring subscriptions that a
 kernel service can search for relevant keys.
 The key service can be configured on by enabling:
 	"Security options"/"Enable access key retention support" (CONFIG_KEYS)
 This document has the following sections:
 	- Key overview
 	- Key service overview
 	- Key access permissions
 	- SELinux support
 	- New procfs files
 	- Userspace system call interface
 	- Kernel services
 	- Notes on accessing payload contents
 	- Defining a key type
 	- Request-key callback service
 	- Key access filesystem
 ============
 KEY OVERVIEW
 ============
 In this context, keys represent units of cryptographic data, authentication
 tokens, keyrings, etc.. These are represented in the kernel by struct key.
 Each key has a number of attributes:
 	- A serial number.
 	- A type.
 	- A description (for matching a key in a search).
 	- Access control information.
 	- An expiry time.
 	- A payload.
 	- State.
 (*) Each key is issued a serial number of type key_serial_t that is unique for
     the lifetime of that key. All serial numbers are positive non-zero 32-bit
     integers.
     Userspace programs can use a key's serial numbers as a way to gain access
     to it, subject to permission checking.
 (*) Each key is of a defined "type". Types must be registered inside the
     kernel by a kernel service (such as a filesystem) before keys of that type
     can be added or used. Userspace programs cannot define new types directly.
     Key types are represented in the kernel by struct key_type. This defines a
     number of operations that can be performed on a key of that type.
     Should a type be removed from the system, all the keys of that type will
     be invalidated.
 (*) Each key has a description. This should be a printable string. The key
     type provides an operation to perform a match between the description on a
     key and a criterion string.
 (*) Each key has an owner user ID, a group ID and a permissions mask. These
     are used to control what a process may do to a key from userspace, and
     whether a kernel service will be able to find the key.
 (*) Each key can be set to expire at a specific time by the key type's
     instantiation function. Keys can also be immortal.
 (*) Each key can have a payload. This is a quantity of data that represent the
     actual "key". In the case of a keyring, this is a list of keys to which
     the keyring links; in the case of a user-defined key, it's an arbitrary
     blob of data.
     Having a payload is not required; and the payload can, in fact, just be a
     value stored in the struct key itself.
     When a key is instantiated, the key type's instantiation function is
     called with a blob of data, and that then creates the key's payload in
     some way.
     Similarly, when userspace wants to read back the contents of the key, if
     permitted, another key type operation will be called to convert the key's
     attached payload back into a blob of data.
 (*) Each key can be in one of a number of basic states:
     (*) Uninstantiated. The key exists, but does not have any data attached.
     	 Keys being requested from userspace will be in this state.
     (*) Instantiated. This is the normal state. The key is fully formed, and
 	 has data attached.
     (*) Negative. This is a relatively short-lived state. The key acts as a
 	 note saying that a previous call out to userspace failed, and acts as
 	 a throttle on key lookups. A negative key can be updated to a normal
 	 state.
     (*) Expired. Keys can have lifetimes set. If their lifetime is exceeded,
 	 they traverse to this state. An expired key can be updated back to a
 	 normal state.
     (*) Revoked. A key is put in this state by userspace action. It can't be
 	 found or operated upon (apart from by unlinking it).
     (*) Dead. The key's type was unregistered, and so the key is now useless.
 ====================
 KEY SERVICE OVERVIEW
 ====================
 The key service provides a number of features besides keys:
 (*) The key service defines two special key types:
     (+) "keyring"
 	 Keyrings are special keys that contain a list of other keys. Keyring
 	 lists can be modified using various system calls. Keyrings should not
 	 be given a payload when created.
     (+) "user"
 	 A key of this type has a description and a payload that are arbitrary
 	 blobs of data. These can be created, updated and read by userspace,
 	 and aren't intended for use by kernel services.
 (*) Each process subscribes to three keyrings: a thread-specific keyring, a
     process-specific keyring, and a session-specific keyring.
     The thread-specific keyring is discarded from the child when any sort of
     clone, fork, vfork or execve occurs. A new keyring is created only when
     required.
     The process-specific keyring is replaced with an empty one in the child on
     clone, fork, vfork unless CLONE_THREAD is supplied, in which case it is
     shared. execve also discards the process's process keyring and creates a
     new one.
     The session-specific keyring is persistent across clone, fork, vfork and
     execve, even when the latter executes a set-UID or set-GID binary. A
     process can, however, replace its current session keyring with a new one
     by using PR_JOIN_SESSION_KEYRING. It is permitted to request an anonymous
     new one, or to attempt to create or join one of a specific name.
     The ownership of the thread keyring changes when the real UID and GID of
     the thread changes.
 (*) Each user ID resident in the system holds two special keyrings: a user
     specific keyring and a default user session keyring. The default session
     keyring is initialised with a link to the user-specific keyring.
     When a process changes its real UID, if it used to have no session key, it
     will be subscribed to the default session key for the new UID.
     If a process attempts to access its session key when it doesn't have one,
     it will be subscribed to the default for its current UID.
 (*) Each user has two quotas against which the keys they own are tracked. One
     limits the total number of keys and keyrings, the other limits the total
     amount of description and payload space that can be consumed.
     The user can view information on this and other statistics through procfs
     files.
     Process-specific and thread-specific keyrings are not counted towards a
     user's quota.
     If a system call that modifies a key or keyring in some way would put the
     user over quota, the operation is refused and error EDQUOT is returned.
 (*) There's a system call interface by which userspace programs can create and
     manipulate keys and keyrings.
 (*) There's a kernel interface by which services can register types and search
     for keys.
 (*) There's a way for the a search done from the kernel to call back to
     userspace to request a key that can't be found in a process's keyrings.
 (*) An optional filesystem is available through which the key database can be
     viewed and manipulated.
 ======================
 KEY ACCESS PERMISSIONS
 ======================
 Keys have an owner user ID, a group access ID, and a permissions mask. The mask
 has up to eight bits each for possessor, user, group and other access. Only
 six of each set of eight bits are defined. These permissions granted are:
 (*) View
     This permits a key or keyring's attributes to be viewed - including key
     type and description.
 (*) Read
     This permits a key's payload to be viewed or a keyring's list of linked
     keys.
 (*) Write
     This permits a key's payload to be instantiated or updated, or it allows a
     link to be added to or removed from a keyring.
 (*) Search
     This permits keyrings to be searched and keys to be found. Searches can
     only recurse into nested keyrings that have search permission set.
 (*) Link
     This permits a key or keyring to be linked to. To create a link from a
     keyring to a key, a process must have Write permission on the keyring and
     Link permission on the key.
 (*) Set Attribute
     This permits a key's UID, GID and permissions mask to be changed.
 For changing the ownership, group ID or permissions mask, being the owner of
 the key or having the sysadmin capability is sufficient.
 ===============
 SELINUX SUPPORT
 ===============
 The security class "key" has been added to SELinux so that mandatory access
 controls can be applied to keys created within various contexts.  This support
 is preliminary, and is likely to change quite significantly in the near future.
 Currently, all of the basic permissions explained above are provided in SELinux
 as well; SELinux is simply invoked after all basic permission checks have been
 performed.
 The value of the file /proc/self/attr/keycreate influences the labeling of
 newly-created keys.  If the contents of that file correspond to an SELinux
 security context, then the key will be assigned that context.  Otherwise, the
 key will be assigned the current context of the task that invoked the key
 creation request.  Tasks must be granted explicit permission to assign a
 particular context to newly-created keys, using the "create" permission in the
 key security class.
 The default keyrings associated with users will be labeled with the default
 context of the user if and only if the login programs have been instrumented to
 properly initialize keycreate during the login process.  Otherwise, they will
 be labeled with the context of the login program itself.
 Note, however, that the default keyrings associated with the root user are
 labeled with the default kernel context, since they are created early in the
 boot process, before root has a chance to log in.
 The keyrings associated with new threads are each labeled with the context of
 their associated thread, and both session and process keyrings are handled
 similarly.
 ================
 NEW PROCFS FILES
 ================
 Two files have been added to procfs by which an administrator can find out
 about the status of the key service:
 (*) /proc/keys
     This lists the keys that are currently viewable by the task reading the
     file, giving information about their type, description and permissions.
     It is not possible to view the payload of the key this way, though some
     information about it may be given.
     The only keys included in the list are those that grant View permission to
     the reading process whether or not it possesses them.  Note that LSM
     security checks are still performed, and may further filter out keys that
     the current process is not authorised to view.
     The contents of the file look like this:
 	SERIAL   FLAGS  USAGE EXPY PERM     UID   GID   TYPE      DESCRIPTION: SUMMARY
 	00000001 I-----    39 perm 1f3f0000     0     0 keyring   _uid_ses.0: 1/4
 	00000002 I-----     2 perm 1f3f0000     0     0 keyring   _uid.0: empty
 	00000007 I-----     1 perm 1f3f0000     0     0 keyring   _pid.1: empty
 	0000018d I-----     1 perm 1f3f0000     0     0 keyring   _pid.412: empty
 	000004d2 I--Q--     1 perm 1f3f0000    32    -1 keyring   _uid.32: 1/4
 	000004d3 I--Q--     3 perm 1f3f0000    32    -1 keyring   _uid_ses.32: empty
 	00000892 I--QU-     1 perm 1f000000     0     0 user      metal:copper: 0
 	00000893 I--Q-N     1  35s 1f3f0000     0     0 user      metal:silver: 0
 	00000894 I--Q--     1  10h 003f0000     0     0 user      metal:gold: 0
     The flags are:
 	I	Instantiated
 	R	Revoked
 	D	Dead
 	Q	Contributes to user's quota
 	U	Under contruction by callback to userspace
 	N	Negative key
     This file must be enabled at kernel configuration time as it allows anyone
     to list the keys database.
 (*) /proc/key-users
     This file lists the tracking data for each user that has at least one key
     on the system.  Such data includes quota information and statistics:
 	[root@andromeda root]# cat /proc/key-users
 	0:     46 45/45 1/100 13/10000
 	29:     2 2/2 2/100 40/10000
 	32:     2 2/2 2/100 40/10000
 	38:     2 2/2 2/100 40/10000
     The format of each line is
 	<UID>:			User ID to which this applies
 	<usage>			Structure refcount
 	<inst>/<keys>		Total number of keys and number instantiated
 	<keys>/<max>		Key count quota
 	<bytes>/<max>		Key size quota
 ===============================
 USERSPACE SYSTEM CALL INTERFACE
 ===============================
 Userspace can manipulate keys directly through three new syscalls: add_key,
 request_key and keyctl. The latter provides a number of functions for
 manipulating keys.
 When referring to a key directly, userspace programs should use the key's
 serial number (a positive 32-bit integer). However, there are some special
 values available for referring to special keys and keyrings that relate to the
 process making the call:
 	CONSTANT			VALUE	KEY REFERENCED
 	==============================	======	===========================
 	KEY_SPEC_THREAD_KEYRING		-1	thread-specific keyring
 	KEY_SPEC_PROCESS_KEYRING	-2	process-specific keyring
 	KEY_SPEC_SESSION_KEYRING	-3	session-specific keyring
 	KEY_SPEC_USER_KEYRING		-4	UID-specific keyring
 	KEY_SPEC_USER_SESSION_KEYRING	-5	UID-session keyring
 	KEY_SPEC_GROUP_KEYRING		-6	GID-specific keyring
 	KEY_SPEC_REQKEY_AUTH_KEY	-7	assumed request_key()
 						  authorisation key
 The main syscalls are:
 (*) Create a new key of given type, description and payload and add it to the
     nominated keyring:
 	key_serial_t add_key(const char *type, const char *desc,
 			     const void *payload, size_t plen,
 			     key_serial_t keyring);
     If a key of the same type and description as that proposed already exists
     in the keyring, this will try to update it with the given payload, or it
     will return error EEXIST if that function is not supported by the key
     type. The process must also have permission to write to the key to be able
     to update it. The new key will have all user permissions granted and no
     group or third party permissions.
     Otherwise, this will attempt to create a new key of the specified type and
     description, and to instantiate it with the supplied payload and attach it
     to the keyring. In this case, an error will be generated if the process
     does not have permission to write to the keyring.
     The payload is optional, and the pointer can be NULL if not required by
     the type. The payload is plen in size, and plen can be zero for an empty
     payload.
     A new keyring can be generated by setting type "keyring", the keyring name
     as the description (or NULL) and setting the payload to NULL.
     User defined keys can be created by specifying type "user". It is
     recommended that a user defined key's description by prefixed with a type
     ID and a colon, such as "krb5tgt:" for a Kerberos 5 ticket granting
     ticket.
     Any other type must have been registered with the kernel in advance by a
     kernel service such as a filesystem.
     The ID of the new or updated key is returned if successful.
 (*) Search the process's keyrings for a key, potentially calling out to
     userspace to create it.
 	key_serial_t request_key(const char *type, const char *description,
 				 const char *callout_info,
 				 key_serial_t dest_keyring);
     This function searches all the process's keyrings in the order thread,
     process, session for a matching key. This works very much like
     KEYCTL_SEARCH, including the optional attachment of the discovered key to
     a keyring.
     If a key cannot be found, and if callout_info is not NULL, then
     /sbin/request-key will be invoked in an attempt to obtain a key. The
     callout_info string will be passed as an argument to the program.
     See also Documentation/keys-request-key.txt.
 The keyctl syscall functions are:
 (*) Map a special key ID to a real key ID for this process:
 	key_serial_t keyctl(KEYCTL_GET_KEYRING_ID, key_serial_t id,
 			    int create);
     The special key specified by "id" is looked up (with the key being created
     if necessary) and the ID of the key or keyring thus found is returned if
     it exists.
     If the key does not yet exist, the key will be created if "create" is
     non-zero; and the error ENOKEY will be returned if "create" is zero.
 (*) Replace the session keyring this process subscribes to with a new one:
 	key_serial_t keyctl(KEYCTL_JOIN_SESSION_KEYRING, const char *name);
     If name is NULL, an anonymous keyring is created attached to the process
     as its session keyring, displacing the old session keyring.
     If name is not NULL, if a keyring of that name exists, the process
     attempts to attach it as the session keyring, returning an error if that
     is not permitted; otherwise a new keyring of that name is created and
     attached as the session keyring.
     To attach to a named keyring, the keyring must have search permission for
     the process's ownership.
     The ID of the new session keyring is returned if successful.
 (*) Update the specified key:
 	long keyctl(KEYCTL_UPDATE, key_serial_t key, const void *payload,
 		    size_t plen);
     This will try to update the specified key with the given payload, or it
     will return error EOPNOTSUPP if that function is not supported by the key
     type. The process must also have permission to write to the key to be able
     to update it.
     The payload is of length plen, and may be absent or empty as for
     add_key().
 (*) Revoke a key:
 	long keyctl(KEYCTL_REVOKE, key_serial_t key);
     This makes a key unavailable for further operations. Further attempts to
     use the key will be met with error EKEYREVOKED, and the key will no longer
     be findable.
 (*) Change the ownership of a key:
 	long keyctl(KEYCTL_CHOWN, key_serial_t key, uid_t uid, gid_t gid);
     This function permits a key's owner and group ID to be changed. Either one
     of uid or gid can be set to -1 to suppress that change.
     Only the superuser can change a key's owner to something other than the
     key's current owner. Similarly, only the superuser can change a key's
     group ID to something other than the calling process's group ID or one of
     its group list members.
 (*) Change the permissions mask on a key:
 	long keyctl(KEYCTL_SETPERM, key_serial_t key, key_perm_t perm);
     This function permits the owner of a key or the superuser to change the
     permissions mask on a key.
     Only bits the available bits are permitted; if any other bits are set,
     error EINVAL will be returned.
 (*) Describe a key:
 	long keyctl(KEYCTL_DESCRIBE, key_serial_t key, char *buffer,
 		    size_t buflen);
     This function returns a summary of the key's attributes (but not its
     payload data) as a string in the buffer provided.
     Unless there's an error, it always returns the amount of data it could
     produce, even if that's too big for the buffer, but it won't copy more
     than requested to userspace. If the buffer pointer is NULL then no copy
     will take place.
     A process must have view permission on the key for this function to be
     successful.
     If successful, a string is placed in the buffer in the following format:
 	<type>;<uid>;<gid>;<perm>;<description>
     Where type and description are strings, uid and gid are decimal, and perm
     is hexadecimal. A NUL character is included at the end of the string if
     the buffer is sufficiently big.
     This can be parsed with
 	sscanf(buffer, "%[^;];%d;%d;%o;%s", type, &uid, &gid, &mode, desc);
 (*) Clear out a keyring:
 	long keyctl(KEYCTL_CLEAR, key_serial_t keyring);
     This function clears the list of keys attached to a keyring. The calling
     process must have write permission on the keyring, and it must be a
     keyring (or else error ENOTDIR will result).
 (*) Link a key into a keyring:
 	long keyctl(KEYCTL_LINK, key_serial_t keyring, key_serial_t key);
     This function creates a link from the keyring to the key. The process must
     have write permission on the keyring and must have link permission on the
     key.
     Should the keyring not be a keyring, error ENOTDIR will result; and if the
     keyring is full, error ENFILE will result.
     The link procedure checks the nesting of the keyrings, returning ELOOP if
     it appears too deep or EDEADLK if the link would introduce a cycle.
     Any links within the keyring to keys that match the new key in terms of
     type and description will be discarded from the keyring as the new one is
     added.
 (*) Unlink a key or keyring from another keyring:
 	long keyctl(KEYCTL_UNLINK, key_serial_t keyring, key_serial_t key);
     This function looks through the keyring for the first link to the
     specified key, and removes it if found. Subsequent links to that key are
     ignored. The process must have write permission on the keyring.
     If the keyring is not a keyring, error ENOTDIR will result; and if the key
     is not present, error ENOENT will be the result.
 (*) Search a keyring tree for a key:
 	key_serial_t keyctl(KEYCTL_SEARCH, key_serial_t keyring,
 			    const char *type, const char *description,
 			    key_serial_t dest_keyring);
     This searches the keyring tree headed by the specified keyring until a key
     is found that matches the type and description criteria. Each keyring is
     checked for keys before recursion into its children occurs.
     The process must have search permission on the top level keyring, or else
     error EACCES will result. Only keyrings that the process has search
     permission on will be recursed into, and only keys and keyrings for which
     a process has search permission can be matched. If the specified keyring
     is not a keyring, ENOTDIR will result.
     If the search succeeds, the function will attempt to link the found key
     into the destination keyring if one is supplied (non-zero ID). All the
     constraints applicable to KEYCTL_LINK apply in this case too.
     Error ENOKEY, EKEYREVOKED or EKEYEXPIRED will be returned if the search
     fails. On success, the resulting key ID will be returned.
 (*) Read the payload data from a key:
 	long keyctl(KEYCTL_READ, key_serial_t keyring, char *buffer,
 		    size_t buflen);
     This function attempts to read the payload data from the specified key
     into the buffer. The process must have read permission on the key to
     succeed.
     The returned data will be processed for presentation by the key type. For
     instance, a keyring will return an array of key_serial_t entries
     representing the IDs of all the keys to which it is subscribed. The user
     defined key type will return its data as is. If a key type does not
     implement this function, error EOPNOTSUPP will result.
     As much of the data as can be fitted into the buffer will be copied to
     userspace if the buffer pointer is not NULL.
     On a successful return, the function will always return the amount of data
     available rather than the amount copied.
 (*) Instantiate a partially constructed key.
 	long keyctl(KEYCTL_INSTANTIATE, key_serial_t key,
 		    const void *payload, size_t plen,
 		    key_serial_t keyring);
     If the kernel calls back to userspace to complete the instantiation of a
     key, userspace should use this call to supply data for the key before the
     invoked process returns, or else the key will be marked negative
     automatically.
     The process must have write access on the key to be able to instantiate
     it, and the key must be uninstantiated.
     If a keyring is specified (non-zero), the key will also be linked into
     that keyring, however all the constraints applying in KEYCTL_LINK apply in
     this case too.
     The payload and plen arguments describe the payload data as for add_key().
 (*) Negatively instantiate a partially constructed key.
 	long keyctl(KEYCTL_NEGATE, key_serial_t key,
 		    unsigned timeout, key_serial_t keyring);
     If the kernel calls back to userspace to complete the instantiation of a
     key, userspace should use this call mark the key as negative before the
     invoked process returns if it is unable to fulfil the request.
     The process must have write access on the key to be able to instantiate
     it, and the key must be uninstantiated.
     If a keyring is specified (non-zero), the key will also be linked into
     that keyring, however all the constraints applying in KEYCTL_LINK apply in
     this case too.
 (*) Set the default request-key destination keyring.
 	long keyctl(KEYCTL_SET_REQKEY_KEYRING, int reqkey_defl);
     This sets the default keyring to which implicitly requested keys will be
     attached for this thread. reqkey_defl should be one of these constants:
 	CONSTANT				VALUE	NEW DEFAULT KEYRING
 	======================================	======	=======================
 	KEY_REQKEY_DEFL_NO_CHANGE		-1	No change
 	KEY_REQKEY_DEFL_DEFAULT			0	Default[1]
 	KEY_REQKEY_DEFL_THREAD_KEYRING		1	Thread keyring
 	KEY_REQKEY_DEFL_PROCESS_KEYRING		2	Process keyring
 	KEY_REQKEY_DEFL_SESSION_KEYRING		3	Session keyring
 	KEY_REQKEY_DEFL_USER_KEYRING		4	User keyring
 	KEY_REQKEY_DEFL_USER_SESSION_KEYRING	5	User session keyring
 	KEY_REQKEY_DEFL_GROUP_KEYRING		6	Group keyring
     The old default will be returned if successful and error EINVAL will be
     returned if reqkey_defl is not one of the above values.
     The default keyring can be overridden by the keyring indicated to the
     request_key() system call.
     Note that this setting is inherited across fork/exec.
-     [1] The default default is: the thread keyring if there is one, otherwise
+     [1] The default is: the thread keyring if there is one, otherwise
     the process keyring if there is one, otherwise the session keyring if
     there is one, otherwise the user default session keyring.
 (*) Set the timeout on a key.
 	long keyctl(KEYCTL_SET_TIMEOUT, key_serial_t key, unsigned timeout);
     This sets or clears the timeout on a key. The timeout can be 0 to clear
     the timeout or a number of seconds to set the expiry time that far into
     the future.
     The process must have attribute modification access on a key to set its
     timeout. Timeouts may not be set with this function on negative, revoked
     or expired keys.
 (*) Assume the authority granted to instantiate a key
 	long keyctl(KEYCTL_ASSUME_AUTHORITY, key_serial_t key);
     This assumes or divests the authority required to instantiate the
     specified key. Authority can only be assumed if the thread has the
     authorisation key associated with the specified key in its keyrings
     somewhere.
     Once authority is assumed, searches for keys will also search the
     requester's keyrings using the requester's security label, UID, GID and
     groups.
     If the requested authority is unavailable, error EPERM will be returned,
     likewise if the authority has been revoked because the target key is
     already instantiated.
     If the specified key is 0, then any assumed authority will be divested.
     The assumed authoritative key is inherited across fork and exec.
 ===============
 KERNEL SERVICES
 ===============
 The kernel services for key management are fairly simple to deal with. They can
 be broken down into two areas: keys and key types.
 Dealing with keys is fairly straightforward. Firstly, the kernel service
 registers its type, then it searches for a key of that type. It should retain
 the key as long as it has need of it, and then it should release it. For a
 filesystem or device file, a search would probably be performed during the open
 call, and the key released upon close. How to deal with conflicting keys due to
 two different users opening the same file is left to the filesystem author to
 solve.
 Note that there are two different types of pointers to keys that may be
 encountered:
 (*) struct key *
     This simply points to the key structure itself. Key structures will be at
     least four-byte aligned.
 (*) key_ref_t
     This is equivalent to a struct key *, but the least significant bit is set
     if the caller "possesses" the key. By "possession" it is meant that the
     calling processes has a searchable link to the key from one of its
     keyrings. There are three functions for dealing with these:
 	key_ref_t make_key_ref(const struct key *key,
 			       unsigned long possession);
 	struct key *key_ref_to_ptr(const key_ref_t key_ref);
 	unsigned long is_key_possessed(const key_ref_t key_ref);
     The first function constructs a key reference from a key pointer and
     possession information (which must be 0 or 1 and not any other value).
     The second function retrieves the key pointer from a reference and the
     third retrieves the possession flag.
 When accessing a key's payload contents, certain precautions must be taken to
 prevent access vs modification races. See the section "Notes on accessing
 payload contents" for more information.
 (*) To search for a key, call:
 	struct key *request_key(const struct key_type *type,
 				const char *description,
 				const char *callout_string);
    This is used to request a key or keyring with a description that matches
    the description specified according to the key type's match function. This
    permits approximate matching to occur. If callout_string is not NULL, then
    /sbin/request-key will be invoked in an attempt to obtain the key from
    userspace. In that case, callout_string will be passed as an argument to
    the program.
    Should the function fail error ENOKEY, EKEYEXPIRED or EKEYREVOKED will be
    returned.
    If successful, the key will have been attached to the default keyring for
    implicitly obtained request-key keys, as set by KEYCTL_SET_REQKEY_KEYRING.
    See also Documentation/keys-request-key.txt.
 (*) To search for a key, passing auxiliary data to the upcaller, call:
 	struct key *request_key_with_auxdata(const struct key_type *type,
 					     const char *description,
 					     const char *callout_string,
 					     void *aux);
    This is identical to request_key(), except that the auxiliary data is
    passed to the key_type->request_key() op if it exists.
 (*) When it is no longer required, the key should be released using:
 	void key_put(struct key *key);
    Or:
 	void key_ref_put(key_ref_t key_ref);
    These can be called from interrupt context. If CONFIG_KEYS is not set then
    the argument will not be parsed.
 (*) Extra references can be made to a key by calling the following function:
 	struct key *key_get(struct key *key);
    These need to be disposed of by calling key_put() when they've been
    finished with. The key pointer passed in will be returned. If the pointer
    is NULL or CONFIG_KEYS is not set then the key will not be dereferenced and
    no increment will take place.
 (*) A key's serial number can be obtained by calling:
 	key_serial_t key_serial(struct key *key);
    If key is NULL or if CONFIG_KEYS is not set then 0 will be returned (in the
    latter case without parsing the argument).
 (*) If a keyring was found in the search, this can be further searched by:
 	key_ref_t keyring_search(key_ref_t keyring_ref,
 				 const struct key_type *type,
 				 const char *description)
    This searches the keyring tree specified for a matching key. Error ENOKEY
    is returned upon failure (use IS_ERR/PTR_ERR to determine). If successful,
    the returned key will need to be released.
    The possession attribute from the keyring reference is used to control
    access through the permissions mask and is propagated to the returned key
    reference pointer if successful.
 (*) To check the validity of a key, this function can be called:
 	int validate_key(struct key *key);
    This checks that the key in question hasn't expired or and hasn't been
    revoked. Should the key be invalid, error EKEYEXPIRED or EKEYREVOKED will
    be returned. If the key is NULL or if CONFIG_KEYS is not set then 0 will be
    returned (in the latter case without parsing the argument).
 (*) To register a key type, the following function should be called:
 	int register_key_type(struct key_type *type);
    This will return error EEXIST if a type of the same name is already
    present.
 (*) To unregister a key type, call:
 	void unregister_key_type(struct key_type *type);
 ===================================
 NOTES ON ACCESSING PAYLOAD CONTENTS
 ===================================
 The simplest payload is just a number in key->payload.value. In this case,
 there's no need to indulge in RCU or locking when accessing the payload.
 More complex payload contents must be allocated and a pointer to them set in
 key->payload.data. One of the following ways must be selected to access the
 data:
 (1) Unmodifiable key type.
     If the key type does not have a modify method, then the key's payload can
     be accessed without any form of locking, provided that it's known to be
     instantiated (uninstantiated keys cannot be "found").
 (2) The key's semaphore.
     The semaphore could be used to govern access to the payload and to control
     the payload pointer. It must be write-locked for modifications and would
     have to be read-locked for general access. The disadvantage of doing this
     is that the accessor may be required to sleep.
 (3) RCU.
     RCU must be used when the semaphore isn't already held; if the semaphore
     is held then the contents can't change under you unexpectedly as the
     semaphore must still be used to serialise modifications to the key. The
     key management code takes care of this for the key type.
     However, this means using:
 	rcu_read_lock() ... rcu_dereference() ... rcu_read_unlock()
     to read the pointer, and:
 	rcu_dereference() ... rcu_assign_pointer() ... call_rcu()
     to set the pointer and dispose of the old contents after a grace period.
     Note that only the key type should ever modify a key's payload.
     Furthermore, an RCU controlled payload must hold a struct rcu_head for the
     use of call_rcu() and, if the payload is of variable size, the length of
     the payload. key->datalen cannot be relied upon to be consistent with the
     payload just dereferenced if the key's semaphore is not held.
 ===================
 DEFINING A KEY TYPE
 ===================
 A kernel service may want to define its own key type. For instance, an AFS
 filesystem might want to define a Kerberos 5 ticket key type. To do this, it
 author fills in a struct key_type and registers it with the system.
 The structure has a number of fields, some of which are mandatory:
 (*) const char *name
     The name of the key type. This is used to translate a key type name
     supplied by userspace into a pointer to the structure.
 (*) size_t def_datalen
     This is optional - it supplies the default payload data length as
     contributed to the quota. If the key type's payload is always or almost
     always the same size, then this is a more efficient way to do things.
     The data length (and quota) on a particular key can always be changed
     during instantiation or update by calling:
 	int key_payload_reserve(struct key *key, size_t datalen);
     With the revised data length. Error EDQUOT will be returned if this is not
     viable.
 (*) int (*instantiate)(struct key *key, const void *data, size_t datalen);
     This method is called to attach a payload to a key during construction.
     The payload attached need not bear any relation to the data passed to this
     function.
     If the amount of data attached to the key differs from the size in
     keytype->def_datalen, then key_payload_reserve() should be called.
     This method does not have to lock the key in order to attach a payload.
     The fact that KEY_FLAG_INSTANTIATED is not set in key->flags prevents
     anything else from gaining access to the key.
     It is safe to sleep in this method.
 (*) int (*update)(struct key *key, const void *data, size_t datalen);
     If this type of key can be updated, then this method should be provided.
     It is called to update a key's payload from the blob of data provided.
     key_payload_reserve() should be called if the data length might change
     before any changes are actually made. Note that if this succeeds, the type
     is committed to changing the key because it's already been altered, so all
     memory allocation must be done first.
     The key will have its semaphore write-locked before this method is called,
     but this only deters other writers; any changes to the key's payload must
     be made under RCU conditions, and call_rcu() must be used to dispose of
     the old payload.
     key_payload_reserve() should be called before the changes are made, but
     after all allocations and other potentially failing function calls are
     made.
     It is safe to sleep in this method.
 (*) int (*match)(const struct key *key, const void *desc);
     This method is called to match a key against a description. It should
     return non-zero if the two match, zero if they don't.
     This method should not need to lock the key in any way. The type and
     description can be considered invariant, and the payload should not be
     accessed (the key may not yet be instantiated).
     It is not safe to sleep in this method; the caller may hold spinlocks.
 (*) void (*revoke)(struct key *key);
     This method is optional.  It is called to discard part of the payload
     data upon a key being revoked.  The caller will have the key semaphore
     write-locked.
     It is safe to sleep in this method, though care should be taken to avoid
     a deadlock against the key semaphore.
 (*) void (*destroy)(struct key *key);
     This method is optional. It is called to discard the payload data on a key
     when it is being destroyed.
     This method does not need to lock the key to access the payload; it can
     consider the key as being inaccessible at this time. Note that the key's
     type may have been changed before this function is called.
     It is not safe to sleep in this method; the caller may hold spinlocks.
 (*) void (*describe)(const struct key *key, struct seq_file *p);
     This method is optional. It is called during /proc/keys reading to
     summarise a key's description and payload in text form.
     This method will be called with the RCU read lock held. rcu_dereference()
     should be used to read the payload pointer if the payload is to be
     accessed. key->datalen cannot be trusted to stay consistent with the
     contents of the payload.
     The description will not change, though the key's state may.
     It is not safe to sleep in this method; the RCU read lock is held by the
     caller.
 (*) long (*read)(const struct key *key, char __user *buffer, size_t buflen);
     This method is optional. It is called by KEYCTL_READ to translate the
     key's payload into something a blob of data for userspace to deal with.
     Ideally, the blob should be in the same format as that passed in to the
     instantiate and update methods.
     If successful, the blob size that could be produced should be returned
     rather than the size copied.
     This method will be called with the key's semaphore read-locked. This will
     prevent the key's payload changing. It is not necessary to use RCU locking
     when accessing the key's payload. It is safe to sleep in this method, such
     as might happen when the userspace buffer is accessed.
 (*) int (*request_key)(struct key *key, struct key *authkey, const char *op,
 			void *aux);
     This method is optional.  If provided, request_key() and
     request_key_with_auxdata() will invoke this function rather than
     upcalling to /sbin/request-key to operate upon a key of this type.
     The aux parameter is as passed to request_key_with_auxdata() or is NULL
     otherwise.  Also passed are the key to be operated upon, the
     authorisation key for this operation and the operation type (currently
     only "create").
     This function should return only when the upcall is complete.  Upon return
     the authorisation key will be revoked, and the target key will be
     negatively instantiated if it is still uninstantiated.  The error will be
     returned to the caller of request_key*().
 ============================
 REQUEST-KEY CALLBACK SERVICE
 ============================
 To create a new key, the kernel will attempt to execute the following command
 line:
 	/sbin/request-key create <key> <uid> <gid> \
 		<threadring> <processring> <sessionring> <callout_info>
 <key> is the key being constructed, and the three keyrings are the process
 keyrings from the process that caused the search to be issued. These are
 included for two reasons:
  (1) There may be an authentication token in one of the keyrings that is
      required to obtain the key, eg: a Kerberos Ticket-Granting Ticket.
  (2) The new key should probably be cached in one of these rings.
 This program should set it UID and GID to those specified before attempting to
 access any more keys. It may then look around for a user specific process to
 hand the request off to (perhaps a path held in placed in another key by, for
 example, the KDE desktop manager).
 The program (or whatever it calls) should finish construction of the key by
 calling KEYCTL_INSTANTIATE, which also permits it to cache the key in one of
 the keyrings (probably the session ring) before returning. Alternatively, the
 key can be marked as negative with KEYCTL_NEGATE; this also permits the key to
 be cached in one of the keyrings.
 If it returns with the key remaining in the unconstructed state, the key will
 be marked as being negative, it will be added to the session keyring, and an
 error will be returned to the key requestor.
 Supplementary information may be provided from whoever or whatever invoked this
 service. This will be passed as the <callout_info> parameter. If no such
 information was made available, then "-" will be passed as this parameter
 instead.
 Similarly, the kernel may attempt to update an expired or a soon to expire key
 by executing:
 	/sbin/request-key update <key> <uid> <gid> \
 		<threadring> <processring> <sessionring>
 In this case, the program isn't required to actually attach the key to a ring;
 the rings are provided for reference.
 				  Command Line Options for Linux/m68k
 				  ===================================
 Last Update: 2 May 1999
 Linux/m68k version: 2.2.6
 Author: Roman.Hodek@informatik.uni-erlangen.de (Roman Hodek)
 Update: jds@kom.auc.dk (Jes Sorensen) and faq@linux-m68k.org (Chris Lawrence)
 0) Introduction
 ===============
  Often I've been asked which command line options the Linux/m68k
 kernel understands, or how the exact syntax for the ... option is, or
 ... about the option ... . I hope, this document supplies all the
 answers...
  Note that some options might be outdated, their descriptions being
 incomplete or missing. Please update the information and send in the
 patches.
 1) Overview of the Kernel's Option Processing
 =============================================
 The kernel knows three kinds of options on its command line:
  1) kernel options
  2) environment settings
  3) arguments for init
 To which of these classes an argument belongs is determined as
 follows: If the option is known to the kernel itself, i.e. if the name
 (the part before the '=') or, in some cases, the whole argument string
 is known to the kernel, it belongs to class 1. Otherwise, if the
 argument contains an '=', it is of class 2, and the definition is put
 into init's environment. All other arguments are passed to init as
 command line options.
  This document describes the valid kernel options for Linux/m68k in
 the version mentioned at the start of this file. Later revisions may
 add new such options, and some may be missing in older versions.
  In general, the value (the part after the '=') of an option is a
 list of values separated by commas. The interpretation of these values
 is up to the driver that "owns" the option. This association of
 options with drivers is also the reason that some are further
 subdivided.
 2) General Kernel Options
 =========================
 2.1) root=
 ----------
 Syntax: root=/dev/<device>
    or: root=<hex_number>
 This tells the kernel which device it should mount as the root
 filesystem. The device must be a block device with a valid filesystem
 on it.
  The first syntax gives the device by name. These names are converted
 into a major/minor number internally in the kernel in an unusual way.
 Normally, this "conversion" is done by the device files in /dev, but
 this isn't possible here, because the root filesystem (with /dev)
 isn't mounted yet... So the kernel parses the name itself, with some
 hardcoded name to number mappings. The name must always be a
 combination of two or three letters, followed by a decimal number.
 Valid names are:
  /dev/ram: -> 0x0100 (initial ramdisk)
  /dev/hda: -> 0x0300 (first IDE disk)
  /dev/hdb: -> 0x0340 (second IDE disk)
  /dev/sda: -> 0x0800 (first SCSI disk)
  /dev/sdb: -> 0x0810 (second SCSI disk)
  /dev/sdc: -> 0x0820 (third SCSI disk)
  /dev/sdd: -> 0x0830 (forth SCSI disk)
  /dev/sde: -> 0x0840 (fifth SCSI disk)
  /dev/fd : -> 0x0200 (floppy disk)
  /dev/xda: -> 0x0c00 (first XT disk, unused in Linux/m68k)
  /dev/xdb: -> 0x0c40 (second XT disk, unused in Linux/m68k)
  /dev/ada: -> 0x1c00 (first ACSI device)
  /dev/adb: -> 0x1c10 (second ACSI device)
  /dev/adc: -> 0x1c20 (third ACSI device)
  /dev/add: -> 0x1c30 (forth ACSI device)
 The last four names are available only if the kernel has been compiled
 with Atari and ACSI support.
  The name must be followed by a decimal number, that stands for the
 partition number. Internally, the value of the number is just
 added to the device number mentioned in the table above. The
 exceptions are /dev/ram and /dev/fd, where /dev/ram refers to an
 initial ramdisk loaded by your bootstrap program (please consult the
 instructions for your bootstrap program to find out how to load an
 initial ramdisk). As of kernel version 2.0.18 you must specify
 /dev/ram as the root device if you want to boot from an initial
 ramdisk. For the floppy devices, /dev/fd, the number stands for the
 floppy drive number (there are no partitions on floppy disks). I.e.,
 /dev/fd0 stands for the first drive, /dev/fd1 for the second, and so
 on. Since the number is just added, you can also force the disk format
 by adding a number greater than 3. If you look into your /dev
 directory, use can see the /dev/fd0D720 has major 2 and minor 16. You
 can specify this device for the root FS by writing "root=/dev/fd16" on
 the kernel command line.
 [Strange and maybe uninteresting stuff ON]
  This unusual translation of device names has some strange
 consequences: If, for example, you have a symbolic link from /dev/fd
 to /dev/fd0D720 as an abbreviation for floppy driver #0 in DD format,
 you cannot use this name for specifying the root device, because the
 kernel cannot see this symlink before mounting the root FS and it
 isn't in the table above. If you use it, the root device will not be 
 set at all, without an error message. Another example: You cannot use a
 partition on e.g. the sixth SCSI disk as the root filesystem, if you
 want to specify it by name. This is, because only the devices up to
 /dev/sde are in the table above, but not /dev/sdf. Although, you can
 use the sixth SCSI disk for the root FS, but you have to specify the
 device by number... (see below). Or, even more strange, you can use the
 fact that there is no range checking of the partition number, and your
 knowledge that each disk uses 16 minors, and write "root=/dev/sde17"
 (for /dev/sdf1).
 [Strange and maybe uninteresting stuff OFF]
  If the device containing your root partition isn't in the table
 above, you can also specify it by major and minor numbers. These are
 written in hex, with no prefix and no separator between. E.g., if you
 have a CD with contents appropriate as a root filesystem in the first
 SCSI CD-ROM drive, you boot from it by "root=0b00". Here, hex "0b" =
 decimal 11 is the major of SCSI CD-ROMs, and the minor 0 stands for
 the first of these. You can find out all valid major numbers by
 looking into include/linux/major.h.
 2.2) ro, rw
 -----------
 Syntax: ro
    or: rw
 These two options tell the kernel whether it should mount the root
 filesystem read-only or read-write. The default is read-only, except
 for ramdisks, which default to read-write.
 2.3) debug
 ----------
 Syntax: debug
 This raises the kernel log level to 10 (the default is 7). This is the
 same level as set by the "dmesg" command, just that the maximum level
 selectable by dmesg is 8.
 2.4) debug=
 -----------
 Syntax: debug=<device>
 This option causes certain kernel messages be printed to the selected
 debugging device. This can aid debugging the kernel, since the
 messages can be captured and analyzed on some other machine. Which
 devices are possible depends on the machine type. There are no checks
 for the validity of the device name. If the device isn't implemented,
 nothing happens.
  Messages logged this way are in general stack dumps after kernel
 memory faults or bad kernel traps, and kernel panics. To be exact: all
 messages of level 0 (panic messages) and all messages printed while
 the log level is 8 or more (their level doesn't matter). Before stack
 dumps, the kernel sets the log level to 10 automatically. A level of
 at least 8 can also be set by the "debug" command line option (see
 2.3) and at run time with "dmesg -n 8".
 Devices possible for Amiga:
 - "ser": built-in serial port; parameters: 9600bps, 8N1
 - "mem": Save the messages to a reserved area in chip mem. After
          rebooting, they can be read under AmigaOS with the tool
          'dmesg'.
 Devices possible for Atari:
 - "ser1": ST-MFP serial port ("Modem1"); parameters: 9600bps, 8N1
 - "ser2": SCC channel B serial port ("Modem2"); parameters: 9600bps, 8N1
 - "ser" : default serial port
           This is "ser2" for a Falcon, and "ser1" for any other machine
 - "midi": The MIDI port; parameters: 31250bps, 8N1
 - "par" : parallel port
           The printing routine for this implements a timeout for the
           case there's no printer connected (else the kernel would
           lock up). The timeout is not exact, but usually a few
           seconds.
 2.6) ramdisk=
 -------------
 Syntax: ramdisk=<size>
  This option instructs the kernel to set up a ramdisk of the given
 size in KBytes. Do not use this option if the ramdisk contents are
 passed by bootstrap! In this case, the size is selected automatically
 and should not be overwritten.
  The only application is for root filesystems on floppy disks, that
 should be loaded into memory. To do that, select the corresponding
 size of the disk as ramdisk size, and set the root device to the disk
 drive (with "root=").
 2.7) swap=
 2.8) buff=
 -----------
  I can't find any sign of these options in 2.2.6.
 3) General Device Options (Amiga and Atari)
 ===========================================
 3.1) ether=
 -----------
 Syntax: ether=[<irq>[,<base_addr>[,<mem_start>[,<mem_end>]]]],<dev-name>
  <dev-name> is the name of a net driver, as specified in
 drivers/net/Space.c in the Linux source. Most prominent are eth0, ...
 eth3, sl0, ... sl3, ppp0, ..., ppp3, dummy, and lo.
  The non-ethernet drivers (sl, ppp, dummy, lo) obviously ignore the
 settings by this options. Also, the existing ethernet drivers for
 Linux/m68k (ariadne, a2065, hydra) don't use them because Zorro boards
 are really Plug-'n-Play, so the "ether=" option is useless altogether
 for Linux/m68k.
 3.2) hd=
 --------
 Syntax: hd=<cylinders>,<heads>,<sectors>
  This option sets the disk geometry of an IDE disk. The first hd=
 option is for the first IDE disk, the second for the second one.
 (I.e., you can give this option twice.) In most cases, you won't have
 to use this option, since the kernel can obtain the geometry data
 itself. It exists just for the case that this fails for one of your
 disks.
 3.3) max_scsi_luns=
 -------------------
 Syntax: max_scsi_luns=<n>
  Sets the maximum number of LUNs (logical units) of SCSI devices to
 be scanned. Valid values for <n> are between 1 and 8. Default is 8 if
 "Probe all LUNs on each SCSI device" was selected during the kernel
 configuration, else 1.
 3.4) st=
 --------
 Syntax: st=<buffer_size>,[<write_thres>,[<max_buffers>]]
  Sets several parameters of the SCSI tape driver. <buffer_size> is
 the number of 512-byte buffers reserved for tape operations for each
 device. <write_thres> sets the number of blocks which must be filled
 to start an actual write operation to the tape. Maximum value is the
 total number of buffers. <max_buffer> limits the total number of
 buffers allocated for all tape devices.
 3.5) dmasound=
 --------------
 Syntax: dmasound=[<buffers>,<buffer-size>[,<catch-radius>]]
  This option controls some configurations of the Linux/m68k DMA sound
 driver (Amiga and Atari): <buffers> is the number of buffers you want
 to use (minimum 4, default 4), <buffer-size> is the size of each
 buffer in kilobytes (minimum 4, default 32) and <catch-radius> says
 how much percent of error will be tolerated when setting a frequency
 (maximum 10, default 0). For example with 3% you can play 8000Hz
 AU-Files on the Falcon with its hardware frequency of 8195Hz and thus
 don't need to expand the sound.
 4) Options for Atari Only
 =========================
 4.1) video=
 -----------
 Syntax: video=<fbname>:<sub-options...>
 The <fbname> parameter specifies the name of the frame buffer,
 eg. most atari users will want to specify `atafb' here. The
 <sub-options> is a comma-separated list of the sub-options listed
 below.
 NB: Please notice that this option was renamed from `atavideo' to
    `video' during the development of the 1.3.x kernels, thus you
    might need to update your boot-scripts if upgrading to 2.x from
    an 1.2.x kernel.
 NBB: The behavior of video= was changed in 2.1.57 so the recommended
 option is to specify the name of the frame buffer.
 4.1.1) Video Mode
 -----------------
 This sub-option may be any of the predefined video modes, as listed
 in atari/atafb.c in the Linux/m68k source tree. The kernel will
 activate the given video mode at boot time and make it the default
 mode, if the hardware allows. Currently defined names are:
 - stlow           : 320x200x4
 - stmid, default5 : 640x200x2
 - sthigh, default4: 640x400x1
 - ttlow           : 320x480x8, TT only
 - ttmid, default1 : 640x480x4, TT only
 - tthigh, default2: 1280x960x1, TT only
 - vga2            : 640x480x1, Falcon only
 - vga4            : 640x480x2, Falcon only
 - vga16, default3 : 640x480x4, Falcon only
 - vga256          : 640x480x8, Falcon only
 - falh2           : 896x608x1, Falcon only
 - falh16          : 896x608x4, Falcon only
  If no video mode is given on the command line, the kernel tries the
 modes names "default<n>" in turn, until one is possible with the
 hardware in use.
  A video mode setting doesn't make sense, if the external driver is
 activated by a "external:" sub-option.
 4.1.2) inverse
 --------------
 Invert the display. This affects both, text (consoles) and graphics
 (X) display. Usually, the background is chosen to be black. With this
 option, you can make the background white.
 4.1.3) font
 -----------
 Syntax: font:<fontname>
 Specify the font to use in text modes. Currently you can choose only
 between `VGA8x8', `VGA8x16' and `PEARL8x8'. `VGA8x8' is default, if the
 vertical size of the display is less than 400 pixel rows. Otherwise, the
 `VGA8x16' font is the default.
 4.1.4) hwscroll_
 ----------------
 Syntax: hwscroll_<n>
 The number of additional lines of video memory to reserve for
 speeding up the scrolling ("hardware scrolling"). Hardware scrolling
 is possible only if the kernel can set the video base address in steps
 fine enough. This is true for STE, MegaSTE, TT, and Falcon. It is not
 possible with plain STs and graphics cards (The former because the
 base address must be on a 256 byte boundary there, the latter because
 the kernel doesn't know how to set the base address at all.)
  By default, <n> is set to the number of visible text lines on the
 display. Thus, the amount of video memory is doubled, compared to no
 hardware scrolling. You can turn off the hardware scrolling altogether
 by setting <n> to 0.
 4.1.5) internal:
 ----------------
 Syntax: internal:<xres>;<yres>[;<xres_max>;<yres_max>;<offset>]
 This option specifies the capabilities of some extended internal video
 hardware, like e.g. OverScan. <xres> and <yres> give the (extended)
 dimensions of the screen.
  If your OverScan needs a black border, you have to write the last
 three arguments of the "internal:". <xres_max> is the maximum line
 length the hardware allows, <yres_max> the maximum number of lines.
 <offset> is the offset of the visible part of the screen memory to its
 physical start, in bytes.
  Often, extended interval video hardware has to be activated somehow.
 For this, see the "sw_*" options below.
 4.1.6) external:
 ----------------
 Syntax:
  external:<xres>;<yres>;<depth>;<org>;<scrmem>[;<scrlen>[;<vgabase>\
           [;<colw>[;<coltype>[;<xres_virtual>]]]]]
 [I had to break this line...]
  This is probably the most complicated parameter... It specifies that
 you have some external video hardware (a graphics board), and how to
 use it under Linux/m68k. The kernel cannot know more about the hardware
 than you tell it here! The kernel also is unable to set or change any
 video modes, since it doesn't know about any board internal. So, you
 have to switch to that video mode before you start Linux, and cannot
 switch to another mode once Linux has started.
  The first 3 parameters of this sub-option should be obvious: <xres>,
 <yres> and <depth> give the dimensions of the screen and the number of
-planes (depth). The depth is is the logarithm to base 2 of the number
+planes (depth). The depth is the logarithm to base 2 of the number
 of colors possible. (Or, the other way round: The number of colors is
 2^depth).
  You have to tell the kernel furthermore how the video memory is
 organized. This is done by a letter as <org> parameter:
 'n': "normal planes", i.e. one whole plane after another
 'i': "interleaved planes", i.e. 16 bit of the first plane, than 16 bit
      of the next, and so on... This mode is used only with the
 	  built-in Atari video modes, I think there is no card that
 	  supports this mode.
 'p': "packed pixels", i.e. <depth> consecutive bits stand for all
 	  planes of one pixel; this is the most common mode for 8 planes
 	  (256 colors) on graphic cards
 't': "true color" (more or less packed pixels, but without a color
 	  lookup table); usually depth is 24
 For monochrome modes (i.e., <depth> is 1), the <org> letter has a
 different meaning:
 'n': normal colors, i.e. 0=white, 1=black
 'i': inverted colors, i.e. 0=black, 1=white
  The next important information about the video hardware is the base
 address of the video memory. That is given in the <scrmem> parameter,
 as a hexadecimal number with a "0x" prefix. You have to find out this
 address in the documentation of your hardware.
  The next parameter, <scrlen>, tells the kernel about the size of the
 video memory. If it's missing, the size is calculated from <xres>,
 <yres>, and <depth>. For now, it is not useful to write a value here.
 It would be used only for hardware scrolling (which isn't possible
 with the external driver, because the kernel cannot set the video base
 address), or for virtual resolutions under X (which the X server
 doesn't support yet). So, it's currently best to leave this field
 empty, either by ending the "external:" after the video address or by
 writing two consecutive semicolons, if you want to give a <vgabase>
 (it is allowed to leave this parameter empty).
  The <vgabase> parameter is optional. If it is not given, the kernel
 cannot read or write any color registers of the video hardware, and
 thus you have to set appropriate colors before you start Linux. But if
 your card is somehow VGA compatible, you can tell the kernel the base
 address of the VGA register set, so it can change the color lookup
 table. You have to look up this address in your board's documentation.
 To avoid misunderstandings: <vgabase> is the _base_ address, i.e. a 4k
 aligned address. For read/writing the color registers, the kernel
 uses the addresses vgabase+0x3c7...vgabase+0x3c9. The <vgabase>
 parameter is written in hexadecimal with a "0x" prefix, just as
 <scrmem>.
  <colw> is meaningful only if <vgabase> is specified. It tells the
 kernel how wide each of the color register is, i.e. the number of bits
 per single color (red/green/blue). Default is 6, another quite usual
 value is 8.
  Also <coltype> is used together with <vgabase>. It tells the kernel
 about the color register model of your gfx board. Currently, the types
 "vga" (which is also the default) and "mv300" (SANG MV300) are
 implemented.
  Parameter <xres_virtual> is required for ProMST or ET4000 cards where
 the physical linelength differs from the visible length. With ProMST, 
 xres_virtual must be set to 2048. For ET4000, xres_virtual depends on the
 initialisation of the video-card.
 If you're missing a corresponding yres_virtual: the external part is legacy,
 therefore we don't support hardware-dependent functions like hardware-scroll,
 panning or blanking.
 4.1.7) eclock:
 --------------
 The external pixel clock attached to the Falcon VIDEL shifter. This
 currently works only with the ScreenWonder!
 4.1.8) monitorcap:
 -------------------
 Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
 This describes the capabilities of a multisync monitor. Don't use it
 with a fixed-frequency monitor! For now, only the Falcon frame buffer
 uses the settings of "monitorcap:".
  <vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
 your monitor can work with, in Hz. <hmin> and <hmax> are the same for
 the horizontal frequency, in kHz.
  The defaults are 58;62;31;32 (VGA compatible).
  The defaults for TV/SC1224/SC1435 cover both PAL and NTSC standards.
 4.1.9) keep
 ------------
 If this option is given, the framebuffer device doesn't do any video
 mode calculations and settings on its own. The only Atari fb device
 that does this currently is the Falcon.
  What you reach with this: Settings for unknown video extensions
 aren't overridden by the driver, so you can still use the mode found
 when booting, when the driver doesn't know to set this mode itself.
 But this also means, that you can't switch video modes anymore...
  An example where you may want to use "keep" is the ScreenBlaster for
 the Falcon.
 4.2) atamouse=
 --------------
 Syntax: atamouse=<x-threshold>,[<y-threshold>]
  With this option, you can set the mouse movement reporting threshold.
 This is the number of pixels of mouse movement that have to accumulate
 before the IKBD sends a new mouse packet to the kernel. Higher values
 reduce the mouse interrupt load and thus reduce the chance of keyboard
 overruns. Lower values give a slightly faster mouse responses and
 slightly better mouse tracking.
  You can set the threshold in x and y separately, but usually this is
 of little practical use. If there's just one number in the option, it
 is used for both dimensions. The default value is 2 for both
 thresholds.
 4.3) ataflop=
 -------------
 Syntax: ataflop=<drive type>[,<trackbuffering>[,<steprateA>[,<steprateB>]]]
   The drive type may be 0, 1, or 2, for DD, HD, and ED, resp. This
   setting affects how many buffers are reserved and which formats are
   probed (see also below). The default is 1 (HD). Only one drive type
   can be selected. If you have two disk drives, select the "better"
   type.
   The second parameter <trackbuffer> tells the kernel whether to use
   track buffering (1) or not (0). The default is machine-dependent:
   no for the Medusa and yes for all others.
   With the two following parameters, you can change the default
   steprate used for drive A and B, resp. 
 4.4) atascsi=
 -------------
 Syntax: atascsi=<can_queue>[,<cmd_per_lun>[,<scat-gat>[,<host-id>[,<tagged>]]]]
  This option sets some parameters for the Atari native SCSI driver.
 Generally, any number of arguments can be omitted from the end. And
 for each of the numbers, a negative value means "use default". The
 defaults depend on whether TT-style or Falcon-style SCSI is used.
 Below, defaults are noted as n/m, where the first value refers to
 TT-SCSI and the latter to Falcon-SCSI. If an illegal value is given
 for one parameter, an error message is printed and that one setting is
 ignored (others aren't affected).
  <can_queue>:
    This is the maximum number of SCSI commands queued internally to the
    Atari SCSI driver. A value of 1 effectively turns off the driver
    internal multitasking (if it causes problems). Legal values are >=
    1. <can_queue> can be as high as you like, but values greater than
    <cmd_per_lun> times the number of SCSI targets (LUNs) you have
    don't make sense. Default: 16/8.
  <cmd_per_lun>:
    Maximum number of SCSI commands issued to the driver for one
    logical unit (LUN, usually one SCSI target). Legal values start
    from 1. If tagged queuing (see below) is not used, values greater
    than 2 don't make sense, but waste memory. Otherwise, the maximum
    is the number of command tags available to the driver (currently
    32). Default: 8/1. (Note: Values > 1 seem to cause problems on a
    Falcon, cause not yet known.)
      The <cmd_per_lun> value at a great part determines the amount of
    memory SCSI reserves for itself. The formula is rather
    complicated, but I can give you some hints:
      no scatter-gather  : cmd_per_lun * 232 bytes
      full scatter-gather: cmd_per_lun * approx. 17 Kbytes
  <scat-gat>:
    Size of the scatter-gather table, i.e. the number of requests
    consecutive on the disk that can be merged into one SCSI command.
    Legal values are between 0 and 255. Default: 255/0. Note: This
    value is forced to 0 on a Falcon, since scatter-gather isn't
    possible with the ST-DMA. Not using scatter-gather hurts
    performance significantly.
  <host-id>:
    The SCSI ID to be used by the initiator (your Atari). This is
    usually 7, the highest possible ID. Every ID on the SCSI bus must
    be unique. Default: determined at run time: If the NV-RAM checksum
    is valid, and bit 7 in byte 30 of the NV-RAM is set, the lower 3
    bits of this byte are used as the host ID. (This method is defined
    by Atari and also used by some TOS HD drivers.) If the above
    isn't given, the default ID is 7. (both, TT and Falcon).
  <tagged>:
    0 means turn off tagged queuing support, all other values > 0 mean
    use tagged queuing for targets that support it. Default: currently
    off, but this may change when tagged queuing handling has been
    proved to be reliable.
    Tagged queuing means that more than one command can be issued to
    one LUN, and the SCSI device itself orders the requests so they
    can be performed in optimal order. Not all SCSI devices support
    tagged queuing (:-().
 4.5 switches=
 -------------
 Syntax: switches=<list of switches>
  With this option you can switch some hardware lines that are often
 used to enable/disable certain hardware extensions. Examples are
 OverScan, overclocking, ...
  The <list of switches> is a comma-separated list of the following
 items:
  ikbd: set RTS of the keyboard ACIA high
  midi: set RTS of the MIDI ACIA high
  snd6: set bit 6 of the PSG port A
  snd7: set bit 6 of the PSG port A
 It doesn't make sense to mention a switch more than once (no
 difference to only once), but you can give as many switches as you
 want to enable different features. The switch lines are set as early
 as possible during kernel initialization (even before determining the
 present hardware.)
  All of the items can also be prefixed with "ov_", i.e. "ov_ikbd",
 "ov_midi", ... These options are meant for switching on an OverScan
 video extension. The difference to the bare option is that the
 switch-on is done after video initialization, and somehow synchronized
 to the HBLANK. A speciality is that ov_ikbd and ov_midi are switched
 off before rebooting, so that OverScan is disabled and TOS boots
 correctly.
  If you give an option both, with and without the "ov_" prefix, the
 earlier initialization ("ov_"-less) takes precedence. But the
 switching-off on reset still happens in this case.
 5) Options for Amiga Only:
 ==========================
 5.1) video=
 -----------
 Syntax: video=<fbname>:<sub-options...>
 The <fbname> parameter specifies the name of the frame buffer, valid
 options are `amifb', `cyber', 'virge', `retz3' and `clgen', provided
 that the respective frame buffer devices have been compiled into the
 kernel (or compiled as loadable modules). The behavior of the <fbname>
 option was changed in 2.1.57 so it is now recommended to specify this
 option.
 The <sub-options> is a comma-separated list of the sub-options listed
 below. This option is organized similar to the Atari version of the
 "video"-option (4.1), but knows fewer sub-options.
 5.1.1) video mode
 -----------------
 Again, similar to the video mode for the Atari (see 4.1.1). Predefined
 modes depend on the used frame buffer device.
 OCS, ECS and AGA machines all use the color frame buffer. The following
 predefined video modes are available:
 NTSC modes:
 - ntsc            : 640x200, 15 kHz, 60 Hz
 - ntsc-lace       : 640x400, 15 kHz, 60 Hz interlaced
 PAL modes:
 - pal             : 640x256, 15 kHz, 50 Hz
 - pal-lace        : 640x512, 15 kHz, 50 Hz interlaced
 ECS modes:
 - multiscan       : 640x480, 29 kHz, 57 Hz
 - multiscan-lace  : 640x960, 29 kHz, 57 Hz interlaced
 - euro36          : 640x200, 15 kHz, 72 Hz
 - euro36-lace     : 640x400, 15 kHz, 72 Hz interlaced
 - euro72          : 640x400, 29 kHz, 68 Hz
 - euro72-lace     : 640x800, 29 kHz, 68 Hz interlaced
 - super72         : 800x300, 23 kHz, 70 Hz
 - super72-lace    : 800x600, 23 kHz, 70 Hz interlaced
 - dblntsc-ff      : 640x400, 27 kHz, 57 Hz
 - dblntsc-lace    : 640x800, 27 kHz, 57 Hz interlaced
 - dblpal-ff       : 640x512, 27 kHz, 47 Hz
 - dblpal-lace     : 640x1024, 27 kHz, 47 Hz interlaced
 - dblntsc         : 640x200, 27 kHz, 57 Hz doublescan
 - dblpal          : 640x256, 27 kHz, 47 Hz doublescan
 VGA modes:
 - vga             : 640x480, 31 kHz, 60 Hz
 - vga70           : 640x400, 31 kHz, 70 Hz
 Please notice that the ECS and VGA modes require either an ECS or AGA
 chipset, and that these modes are limited to 2-bit color for the ECS
 chipset and 8-bit color for the AGA chipset.
 5.1.2) depth
 ------------
 Syntax: depth:<nr. of bit-planes>
 Specify the number of bit-planes for the selected video-mode.
 5.1.3) inverse
 --------------
 Use inverted display (black on white). Functionally the same as the
 "inverse" sub-option for the Atari.
 5.1.4) font
 -----------
 Syntax: font:<fontname>
 Specify the font to use in text modes. Functionally the same as the
 "font" sub-option for the Atari, except that `PEARL8x8' is used instead
 of `VGA8x8' if the vertical size of the display is less than 400 pixel
 rows.
 5.1.5) monitorcap:
 -------------------
 Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
 This describes the capabilities of a multisync monitor. For now, only
 the color frame buffer uses the settings of "monitorcap:".
  <vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
 your monitor can work with, in Hz. <hmin> and <hmax> are the same for
 the horizontal frequency, in kHz.
  The defaults are 50;90;15;38 (Generic Amiga multisync monitor).
 5.2) fd_def_df0=
 ----------------
 Syntax: fd_def_df0=<value>
 Sets the df0 value for "silent" floppy drives. The value should be in
 hexadecimal with "0x" prefix.
 5.3) wd33c93=
 -------------
 Syntax: wd33c93=<sub-options...>
 These options affect the A590/A2091, A3000 and GVP Series II SCSI
 controllers.
 The <sub-options> is a comma-separated list of the sub-options listed
 below.
 5.3.1) nosync
 -------------
 Syntax: nosync:bitmask
  bitmask is a byte where the 1st 7 bits correspond with the 7
 possible SCSI devices. Set a bit to prevent sync negotiation on that
 device. To maintain backwards compatibility, a command-line such as
 "wd33c93=255" will be automatically translated to
 "wd33c93=nosync:0xff". The default is to disable sync negotiation for
 all devices, eg. nosync:0xff.
 5.3.2) period
 -------------
 Syntax: period:ns
  `ns' is the minimum # of nanoseconds in a SCSI data transfer
 period. Default is 500; acceptable values are 250 - 1000.
 5.3.3) disconnect
 -----------------
 Syntax: disconnect:x
  Specify x = 0 to never allow disconnects, 2 to always allow them.
 x = 1 does 'adaptive' disconnects, which is the default and generally
 the best choice.
 5.3.4) debug
 ------------
 Syntax: debug:x
  If `DEBUGGING_ON' is defined, x is a bit mask that causes various
 types of debug output to printed - see the DB_xxx defines in
 wd33c93.h.
 5.3.5) clock
 ------------
 Syntax: clock:x
  x = clock input in MHz for WD33c93 chip. Normal values would be from
 8 through 20. The default value depends on your hostadapter(s),
 default for the A3000 internal controller is 14, for the A2091 it's 8
 and for the GVP hostadapters it's either 8 or 14, depending on the
 hostadapter and the SCSI-clock jumper present on some GVP
 hostadapters.
 5.3.6) next
 -----------
  No argument. Used to separate blocks of keywords when there's more
 than one wd33c93-based host adapter in the system.
 5.3.7) nodma
 ------------
 Syntax: nodma:x
  If x is 1 (or if the option is just written as "nodma"), the WD33c93
 controller will not use DMA (= direct memory access) to access the
 Amiga's memory.  This is useful for some systems (like A3000's and
 A4000's with the A3640 accelerator, revision 3.0) that have problems
 using DMA to chip memory.  The default is 0, i.e. to use DMA if
 possible.
 5.4) gvp11=
 -----------
 Syntax: gvp11=<addr-mask>
  The earlier versions of the GVP driver did not handle DMA
 address-mask settings correctly which made it necessary for some
 people to use this option, in order to get their GVP controller
 running under Linux. These problems have hopefully been solved and the
 use of this option is now highly unrecommended!
  Incorrect use can lead to unpredictable behavior, so please only use
 this option if you *know* what you are doing and have a reason to do
 so. In any case if you experience problems and need to use this
 option, please inform us about it by mailing to the Linux/68k kernel
 mailing list.
  The address mask set by this option specifies which addresses are
 valid for DMA with the GVP Series II SCSI controller. An address is
 valid, if no bits are set except the bits that are set in the mask,
 too.
  Some versions of the GVP can only DMA into a 24 bit address range,
 some can address a 25 bit address range while others can use the whole
 32 bit address range for DMA. The correct setting depends on your
 controller and should be autodetected by the driver. An example is the
 24 bit region which is specified by a mask of 0x00fffffe.
 5.5) 53c7xx=
 ------------
 Syntax: 53c7xx=<sub-options...>
 These options affect the A4000T, A4091, WarpEngine, Blizzard 603e+,
 and GForce 040/060 SCSI controllers on the Amiga, as well as the
 builtin MVME 16x SCSI controller.
 The <sub-options> is a comma-separated list of the sub-options listed
 below.
 5.5.1) nosync
 -------------
 Syntax: nosync:0
  Disables sync negotiation for all devices.  Any value after the
  colon is acceptable (and has the same effect).
 5.5.2) noasync
 --------------
 Syntax: noasync:0
  Disables async and sync negotiation for all devices.  Any value
  after the colon is acceptable (and has the same effect).
 5.5.3) nodisconnect
 -------------------
 Syntax: nodisconnect:0
  Disables SCSI disconnects.  Any value after the colon is acceptable
  (and has the same effect).
 5.5.4) validids
 ---------------
 Syntax: validids:0xNN
  Specify which SCSI ids the driver should pay attention to.  This is
  a bitmask (i.e. to only pay attention to ID#4, you'd use 0x10).
  Default is 0x7f (devices 0-6).
 5.5.5) opthi
 5.5.6) optlo
 ------------
 Syntax: opthi:M,optlo:N
  Specify options for "hostdata->options".  The acceptable definitions
  are listed in drivers/scsi/53c7xx.h; the 32 high bits should be in
  opthi and the 32 low bits in optlo.  They must be specified in the
  order opthi=M,optlo=N.
 5.5.7) next
 -----------
  No argument. Used to separate blocks of keywords when there's more
  than one 53c7xx host adapter in the system.
 /* Local Variables: */
 /* mode: text       */
 /* End:             */
 			 ============================
 			 LINUX KERNEL MEMORY BARRIERS
 			 ============================
 By: David Howells <dhowells@redhat.com>
 Contents:
 (*) Abstract memory access model.
     - Device operations.
     - Guarantees.
 (*) What are memory barriers?
     - Varieties of memory barrier.
     - What may not be assumed about memory barriers?
     - Data dependency barriers.
     - Control dependencies.
     - SMP barrier pairing.
     - Examples of memory barrier sequences.
     - Read memory barriers vs load speculation.
 (*) Explicit kernel barriers.
     - Compiler barrier.
     - The CPU memory barriers.
     - MMIO write barrier.
 (*) Implicit kernel memory barriers.
     - Locking functions.
     - Interrupt disabling functions.
     - Miscellaneous functions.
 (*) Inter-CPU locking barrier effects.
     - Locks vs memory accesses.
     - Locks vs I/O accesses.
 (*) Where are memory barriers needed?
     - Interprocessor interaction.
     - Atomic operations.
     - Accessing devices.
     - Interrupts.
 (*) Kernel I/O barrier effects.
 (*) Assumed minimum execution ordering model.
 (*) The effects of the cpu cache.
     - Cache coherency.
     - Cache coherency vs DMA.
     - Cache coherency vs MMIO.
 (*) The things CPUs get up to.
     - And then there's the Alpha.
 (*) References.
 ============================
 ABSTRACT MEMORY ACCESS MODEL
 ============================
 Consider the following abstract model of the system:
 		            :                :
 		            :                :
 		            :                :
 		+-------+   :   +--------+   :   +-------+
 		|       |   :   |        |   :   |       |
 		|       |   :   |        |   :   |       |
 		| CPU 1 |<----->| Memory |<----->| CPU 2 |
 		|       |   :   |        |   :   |       |
 		|       |   :   |        |   :   |       |
 		+-------+   :   +--------+   :   +-------+
 		    ^       :       ^        :       ^
 		    |       :       |        :       |
 		    |       :       |        :       |
 		    |       :       v        :       |
 		    |       :   +--------+   :       |
 		    |       :   |        |   :       |
 		    |       :   |        |   :       |
 		    +---------->| Device |<----------+
 		            :   |        |   :
 		            :   |        |   :
 		            :   +--------+   :
 		            :                :
 Each CPU executes a program that generates memory access operations.  In the
 abstract CPU, memory operation ordering is very relaxed, and a CPU may actually
 perform the memory operations in any order it likes, provided program causality
 appears to be maintained.  Similarly, the compiler may also arrange the
 instructions it emits in any order it likes, provided it doesn't affect the
 apparent operation of the program.
 So in the above diagram, the effects of the memory operations performed by a
 CPU are perceived by the rest of the system as the operations cross the
 interface between the CPU and rest of the system (the dotted lines).
 For example, consider the following sequence of events:
 	CPU 1		CPU 2
 	===============	===============
 	{ A == 1; B == 2 }
 	A = 3;		x = A;
 	B = 4;		y = B;
 The set of accesses as seen by the memory system in the middle can be arranged
 in 24 different combinations:
 	STORE A=3,	STORE B=4,	x=LOAD A->3,	y=LOAD B->4
 	STORE A=3,	STORE B=4,	y=LOAD B->4,	x=LOAD A->3
 	STORE A=3,	x=LOAD A->3,	STORE B=4,	y=LOAD B->4
 	STORE A=3,	x=LOAD A->3,	y=LOAD B->2,	STORE B=4
 	STORE A=3,	y=LOAD B->2,	STORE B=4,	x=LOAD A->3
 	STORE A=3,	y=LOAD B->2,	x=LOAD A->3,	STORE B=4
 	STORE B=4,	STORE A=3,	x=LOAD A->3,	y=LOAD B->4
 	STORE B=4, ...
 	...
 and can thus result in four different combinations of values:
 	x == 1, y == 2
 	x == 1, y == 4
 	x == 3, y == 2
 	x == 3, y == 4
 Furthermore, the stores committed by a CPU to the memory system may not be
 perceived by the loads made by another CPU in the same order as the stores were
 committed.
 As a further example, consider this sequence of events:
 	CPU 1		CPU 2
 	===============	===============
 	{ A == 1, B == 2, C = 3, P == &A, Q == &C }
 	B = 4;		Q = P;
 	P = &B		D = *Q;
 There is an obvious data dependency here, as the value loaded into D depends on
 the address retrieved from P by CPU 2.  At the end of the sequence, any of the
 following results are possible:
 	(Q == &A) and (D == 1)
 	(Q == &B) and (D == 2)
 	(Q == &B) and (D == 4)
 Note that CPU 2 will never try and load C into D because the CPU will load P
 into Q before issuing the load of *Q.
 DEVICE OPERATIONS
 -----------------
 Some devices present their control interfaces as collections of memory
 locations, but the order in which the control registers are accessed is very
 important.  For instance, imagine an ethernet card with a set of internal
 registers that are accessed through an address port register (A) and a data
 port register (D).  To read internal register 5, the following code might then
 be used:
 	*A = 5;
 	x = *D;
 but this might show up as either of the following two sequences:
 	STORE *A = 5, x = LOAD *D
 	x = LOAD *D, STORE *A = 5
 the second of which will almost certainly result in a malfunction, since it set
 the address _after_ attempting to read the register.
 GUARANTEES
 ----------
 There are some minimal guarantees that may be expected of a CPU:
 (*) On any given CPU, dependent memory accesses will be issued in order, with
     respect to itself.  This means that for:
 	Q = P; D = *Q;
     the CPU will issue the following memory operations:
 	Q = LOAD P, D = LOAD *Q
     and always in that order.
 (*) Overlapping loads and stores within a particular CPU will appear to be
     ordered within that CPU.  This means that for:
 	a = *X; *X = b;
     the CPU will only issue the following sequence of memory operations:
 	a = LOAD *X, STORE *X = b
     And for:
 	*X = c; d = *X;
     the CPU will only issue:
 	STORE *X = c, d = LOAD *X
     (Loads and stores overlap if they are targetted at overlapping pieces of
     memory).
 And there are a number of things that _must_ or _must_not_ be assumed:
 (*) It _must_not_ be assumed that independent loads and stores will be issued
     in the order given.  This means that for:
 	X = *A; Y = *B; *D = Z;
     we may get any of the following sequences:
 	X = LOAD *A,  Y = LOAD *B,  STORE *D = Z
 	X = LOAD *A,  STORE *D = Z, Y = LOAD *B
 	Y = LOAD *B,  X = LOAD *A,  STORE *D = Z
 	Y = LOAD *B,  STORE *D = Z, X = LOAD *A
 	STORE *D = Z, X = LOAD *A,  Y = LOAD *B
 	STORE *D = Z, Y = LOAD *B,  X = LOAD *A
 (*) It _must_ be assumed that overlapping memory accesses may be merged or
     discarded.  This means that for:
 	X = *A; Y = *(A + 4);
     we may get any one of the following sequences:
 	X = LOAD *A; Y = LOAD *(A + 4);
 	Y = LOAD *(A + 4); X = LOAD *A;
 	{X, Y} = LOAD {*A, *(A + 4) };
     And for:
 	*A = X; Y = *A;
     we may get either of:
 	STORE *A = X; Y = LOAD *A;
 	STORE *A = Y = X;
 =========================
 WHAT ARE MEMORY BARRIERS?
 =========================
 As can be seen above, independent memory operations are effectively performed
 in random order, but this can be a problem for CPU-CPU interaction and for I/O.
 What is required is some way of intervening to instruct the compiler and the
 CPU to restrict the order.
 Memory barriers are such interventions.  They impose a perceived partial
 ordering over the memory operations on either side of the barrier.
 Such enforcement is important because the CPUs and other devices in a system
 can use a variety of tricks to improve performance - including reordering,
 deferral and combination of memory operations; speculative loads; speculative
 branch prediction and various types of caching.  Memory barriers are used to
 override or suppress these tricks, allowing the code to sanely control the
 interaction of multiple CPUs and/or devices.
 VARIETIES OF MEMORY BARRIER
 ---------------------------
 Memory barriers come in four basic varieties:
 (1) Write (or store) memory barriers.
     A write memory barrier gives a guarantee that all the STORE operations
     specified before the barrier will appear to happen before all the STORE
     operations specified after the barrier with respect to the other
     components of the system.
     A write barrier is a partial ordering on stores only; it is not required
     to have any effect on loads.
     A CPU can be viewed as committing a sequence of store operations to the
     memory system as time progresses.  All stores before a write barrier will
     occur in the sequence _before_ all the stores after the write barrier.
     [!] Note that write barriers should normally be paired with read or data
     dependency barriers; see the "SMP barrier pairing" subsection.
 (2) Data dependency barriers.
     A data dependency barrier is a weaker form of read barrier.  In the case
     where two loads are performed such that the second depends on the result
     of the first (eg: the first load retrieves the address to which the second
     load will be directed), a data dependency barrier would be required to
     make sure that the target of the second load is updated before the address
     obtained by the first load is accessed.
     A data dependency barrier is a partial ordering on interdependent loads
     only; it is not required to have any effect on stores, independent loads
     or overlapping loads.
     As mentioned in (1), the other CPUs in the system can be viewed as
     committing sequences of stores to the memory system that the CPU being
     considered can then perceive.  A data dependency barrier issued by the CPU
     under consideration guarantees that for any load preceding it, if that
     load touches one of a sequence of stores from another CPU, then by the
     time the barrier completes, the effects of all the stores prior to that
     touched by the load will be perceptible to any loads issued after the data
     dependency barrier.
     See the "Examples of memory barrier sequences" subsection for diagrams
     showing the ordering constraints.
     [!] Note that the first load really has to have a _data_ dependency and
     not a control dependency.  If the address for the second load is dependent
     on the first load, but the dependency is through a conditional rather than
     actually loading the address itself, then it's a _control_ dependency and
     a full read barrier or better is required.  See the "Control dependencies"
     subsection for more information.
     [!] Note that data dependency barriers should normally be paired with
     write barriers; see the "SMP barrier pairing" subsection.
 (3) Read (or load) memory barriers.
     A read barrier is a data dependency barrier plus a guarantee that all the
     LOAD operations specified before the barrier will appear to happen before
     all the LOAD operations specified after the barrier with respect to the
     other components of the system.
     A read barrier is a partial ordering on loads only; it is not required to
     have any effect on stores.
     Read memory barriers imply data dependency barriers, and so can substitute
     for them.
     [!] Note that read barriers should normally be paired with write barriers;
     see the "SMP barrier pairing" subsection.
 (4) General memory barriers.
     A general memory barrier gives a guarantee that all the LOAD and STORE
     operations specified before the barrier will appear to happen before all
     the LOAD and STORE operations specified after the barrier with respect to
     the other components of the system.
     A general memory barrier is a partial ordering over both loads and stores.
     General memory barriers imply both read and write memory barriers, and so
     can substitute for either.
 And a couple of implicit varieties:
 (5) LOCK operations.
     This acts as a one-way permeable barrier.  It guarantees that all memory
     operations after the LOCK operation will appear to happen after the LOCK
     operation with respect to the other components of the system.
     Memory operations that occur before a LOCK operation may appear to happen
     after it completes.
     A LOCK operation should almost always be paired with an UNLOCK operation.
 (6) UNLOCK operations.
     This also acts as a one-way permeable barrier.  It guarantees that all
     memory operations before the UNLOCK operation will appear to happen before
     the UNLOCK operation with respect to the other components of the system.
     Memory operations that occur after an UNLOCK operation may appear to
     happen before it completes.
     LOCK and UNLOCK operations are guaranteed to appear with respect to each
     other strictly in the order specified.
     The use of LOCK and UNLOCK operations generally precludes the need for
     other sorts of memory barrier (but note the exceptions mentioned in the
     subsection "MMIO write barrier").
 Memory barriers are only required where there's a possibility of interaction
 between two CPUs or between a CPU and a device.  If it can be guaranteed that
 there won't be any such interaction in any particular piece of code, then
 memory barriers are unnecessary in that piece of code.
 Note that these are the _minimum_ guarantees.  Different architectures may give
 more substantial guarantees, but they may _not_ be relied upon outside of arch
 specific code.
 WHAT MAY NOT BE ASSUMED ABOUT MEMORY BARRIERS?
 ----------------------------------------------
 There are certain things that the Linux kernel memory barriers do not guarantee:
 (*) There is no guarantee that any of the memory accesses specified before a
     memory barrier will be _complete_ by the completion of a memory barrier
     instruction; the barrier can be considered to draw a line in that CPU's
     access queue that accesses of the appropriate type may not cross.
 (*) There is no guarantee that issuing a memory barrier on one CPU will have
     any direct effect on another CPU or any other hardware in the system.  The
     indirect effect will be the order in which the second CPU sees the effects
     of the first CPU's accesses occur, but see the next point:
 (*) There is no guarantee that a CPU will see the correct order of effects
     from a second CPU's accesses, even _if_ the second CPU uses a memory
     barrier, unless the first CPU _also_ uses a matching memory barrier (see
     the subsection on "SMP Barrier Pairing").
 (*) There is no guarantee that some intervening piece of off-the-CPU
     hardware[*] will not reorder the memory accesses.  CPU cache coherency
     mechanisms should propagate the indirect effects of a memory barrier
     between CPUs, but might not do so in order.
 	[*] For information on bus mastering DMA and coherency please read:
 	    Documentation/pci.txt
 	    Documentation/DMA-mapping.txt
 	    Documentation/DMA-API.txt
 DATA DEPENDENCY BARRIERS
 ------------------------
 The usage requirements of data dependency barriers are a little subtle, and
 it's not always obvious that they're needed.  To illustrate, consider the
 following sequence of events:
 	CPU 1		CPU 2
 	===============	===============
 	{ A == 1, B == 2, C = 3, P == &A, Q == &C }
 	B = 4;
 	<write barrier>
 	P = &B
 			Q = P;
 			D = *Q;
 There's a clear data dependency here, and it would seem that by the end of the
 sequence, Q must be either &A or &B, and that:
 	(Q == &A) implies (D == 1)
 	(Q == &B) implies (D == 4)
 But! CPU 2's perception of P may be updated _before_ its perception of B, thus
 leading to the following situation:
 	(Q == &B) and (D == 2) ????
 Whilst this may seem like a failure of coherency or causality maintenance, it
 isn't, and this behaviour can be observed on certain real CPUs (such as the DEC
 Alpha).
 To deal with this, a data dependency barrier or better must be inserted
 between the address load and the data load:
 	CPU 1		CPU 2
 	===============	===============
 	{ A == 1, B == 2, C = 3, P == &A, Q == &C }
 	B = 4;
 	<write barrier>
 	P = &B
 			Q = P;
 			<data dependency barrier>
 			D = *Q;
 This enforces the occurrence of one of the two implications, and prevents the
 third possibility from arising.
 [!] Note that this extremely counterintuitive situation arises most easily on
 machines with split caches, so that, for example, one cache bank processes
 even-numbered cache lines and the other bank processes odd-numbered cache
 lines.  The pointer P might be stored in an odd-numbered cache line, and the
 variable B might be stored in an even-numbered cache line.  Then, if the
 even-numbered bank of the reading CPU's cache is extremely busy while the
 odd-numbered bank is idle, one can see the new value of the pointer P (&B),
 but the old value of the variable B (2).
 Another example of where data dependency barriers might by required is where a
 number is read from memory and then used to calculate the index for an array
 access:
 	CPU 1		CPU 2
 	===============	===============
 	{ M[0] == 1, M[1] == 2, M[3] = 3, P == 0, Q == 3 }
 	M[1] = 4;
 	<write barrier>
 	P = 1
 			Q = P;
 			<data dependency barrier>
 			D = M[Q];
 The data dependency barrier is very important to the RCU system, for example.
 See rcu_dereference() in include/linux/rcupdate.h.  This permits the current
 target of an RCU'd pointer to be replaced with a new modified target, without
 the replacement target appearing to be incompletely initialised.
 See also the subsection on "Cache Coherency" for a more thorough example.
 CONTROL DEPENDENCIES
 --------------------
 A control dependency requires a full read memory barrier, not simply a data
 dependency barrier to make it work correctly.  Consider the following bit of
 code:
 	q = &a;
 	if (p)
 		q = &b;
 	<data dependency barrier>
 	x = *q;
 This will not have the desired effect because there is no actual data
 dependency, but rather a control dependency that the CPU may short-circuit by
 attempting to predict the outcome in advance.  In such a case what's actually
 required is:
 	q = &a;
 	if (p)
 		q = &b;
 	<read barrier>
 	x = *q;
 SMP BARRIER PAIRING
 -------------------
 When dealing with CPU-CPU interactions, certain types of memory barrier should
 always be paired.  A lack of appropriate pairing is almost certainly an error.
 A write barrier should always be paired with a data dependency barrier or read
 barrier, though a general barrier would also be viable.  Similarly a read
 barrier or a data dependency barrier should always be paired with at least an
 write barrier, though, again, a general barrier is viable:
 	CPU 1		CPU 2
 	===============	===============
 	a = 1;
 	<write barrier>
 	b = 2;		x = b;
 			<read barrier>
 			y = a;
 Or:
 	CPU 1		CPU 2
 	===============	===============================
 	a = 1;
 	<write barrier>
 	b = &a;		x = b;
 			<data dependency barrier>
 			y = *x;
 Basically, the read barrier always has to be there, even though it can be of
 the "weaker" type.
 [!] Note that the stores before the write barrier would normally be expected to
 match the loads after the read barrier or data dependency barrier, and vice
 versa:
 	CPU 1                           CPU 2
 	===============                 ===============
 	a = 1;           }----   --->{  v = c
 	b = 2;           }    \ /    {  w = d
 	<write barrier>        \        <read barrier>
 	c = 3;           }    / \    {  x = a;
 	d = 4;           }----   --->{  y = b;
 EXAMPLES OF MEMORY BARRIER SEQUENCES
 ------------------------------------
 Firstly, write barriers act as a partial orderings on store operations.
 Consider the following sequence of events:
 	CPU 1
 	=======================
 	STORE A = 1
 	STORE B = 2
 	STORE C = 3
 	<write barrier>
 	STORE D = 4
 	STORE E = 5
 This sequence of events is committed to the memory coherence system in an order
 that the rest of the system might perceive as the unordered set of { STORE A,
 STORE B, STORE C } all occurring before the unordered set of { STORE D, STORE E
 }:
 	+-------+       :      :
 	|       |       +------+
 	|       |------>| C=3  |     }     /\
 	|       |  :    +------+     }-----  \  -----> Events perceptible
 	|       |  :    | A=1  |     }        \/       to rest of system
 	|       |  :    +------+     }
 	| CPU 1 |  :    | B=2  |     }
 	|       |       +------+     }
 	|       |   wwwwwwwwwwwwwwww }   <--- At this point the write barrier
 	|       |       +------+     }        requires all stores prior to the
 	|       |  :    | E=5  |     }        barrier to be committed before
 	|       |  :    +------+     }        further stores may be take place.
 	|       |------>| D=4  |     }
 	|       |       +------+
 	+-------+       :      :
 	                   |
 	                   | Sequence in which stores are committed to the
 	                   | memory system by CPU 1
 	                   V
 Secondly, data dependency barriers act as a partial orderings on data-dependent
 loads.  Consider the following sequence of events:
 	CPU 1			CPU 2
 	=======================	=======================
 		{ B = 7; X = 9; Y = 8; C = &Y }
 	STORE A = 1
 	STORE B = 2
 	<write barrier>
 	STORE C = &B		LOAD X
 	STORE D = 4		LOAD C (gets &B)
 				LOAD *C (reads B)
 Without intervention, CPU 2 may perceive the events on CPU 1 in some
 effectively random order, despite the write barrier issued by CPU 1:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+  | Sequence of update
 	|       |------>| B=2  |-----       --->| Y->8  |  | of perception on
 	|       |  :    +------+     \          +-------+  | CPU 2
 	| CPU 1 |  :    | A=1  |      \     --->| C->&Y |  V
 	|       |       +------+       |        +-------+
 	|       |   wwwwwwwwwwwwwwww   |        :       :
 	|       |       +------+       |        :       :
 	|       |  :    | C=&B |---    |        :       :       +-------+
 	|       |  :    +------+   \   |        +-------+       |       |
 	|       |------>| D=4  |    ----------->| C->&B |------>|       |
 	|       |       +------+       |        +-------+       |       |
 	+-------+       :      :       |        :       :       |       |
 	                               |        :       :       |       |
 	                               |        :       :       | CPU 2 |
 	                               |        +-------+       |       |
 	    Apparently incorrect --->  |        | B->7  |------>|       |
 	    perception of B (!)        |        +-------+       |       |
 	                               |        :       :       |       |
 	                               |        +-------+       |       |
 	    The load of X holds --->    \       | X->9  |------>|       |
 	    up the maintenance           \      +-------+       |       |
 	    of coherence of B             ----->| B->2  |       +-------+
 	                                        +-------+
 	                                        :       :
 In the above example, CPU 2 perceives that B is 7, despite the load of *C
-(which would be B) coming after the the LOAD of C.
+(which would be B) coming after the LOAD of C.
 If, however, a data dependency barrier were to be placed between the load of C
 and the load of *C (ie: B) on CPU 2:
 	CPU 1			CPU 2
 	=======================	=======================
 		{ B = 7; X = 9; Y = 8; C = &Y }
 	STORE A = 1
 	STORE B = 2
 	<write barrier>
 	STORE C = &B		LOAD X
 	STORE D = 4		LOAD C (gets &B)
 				<data dependency barrier>
 				LOAD *C (reads B)
 then the following will occur:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+
 	|       |------>| B=2  |-----       --->| Y->8  |
 	|       |  :    +------+     \          +-------+
 	| CPU 1 |  :    | A=1  |      \     --->| C->&Y |
 	|       |       +------+       |        +-------+
 	|       |   wwwwwwwwwwwwwwww   |        :       :
 	|       |       +------+       |        :       :
 	|       |  :    | C=&B |---    |        :       :       +-------+
 	|       |  :    +------+   \   |        +-------+       |       |
 	|       |------>| D=4  |    ----------->| C->&B |------>|       |
 	|       |       +------+       |        +-------+       |       |
 	+-------+       :      :       |        :       :       |       |
 	                               |        :       :       |       |
 	                               |        :       :       | CPU 2 |
 	                               |        +-------+       |       |
 	                               |        | X->9  |------>|       |
 	                               |        +-------+       |       |
 	  Makes sure all effects --->   \   ddddddddddddddddd   |       |
 	  prior to the store of C        \      +-------+       |       |
 	  are perceptible to              ----->| B->2  |------>|       |
 	  subsequent loads                      +-------+       |       |
 	                                        :       :       +-------+
 And thirdly, a read barrier acts as a partial order on loads.  Consider the
 following sequence of events:
 	CPU 1			CPU 2
 	=======================	=======================
 		{ A = 0, B = 9 }
 	STORE A=1
 	<write barrier>
 	STORE B=2
 				LOAD B
 				LOAD A
 Without intervention, CPU 2 may then choose to perceive the events on CPU 1 in
 some effectively random order, despite the write barrier issued by CPU 1:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+
 	|       |------>| A=1  |------      --->| A->0  |
 	|       |       +------+      \         +-------+
 	| CPU 1 |   wwwwwwwwwwwwwwww   \    --->| B->9  |
 	|       |       +------+        |       +-------+
 	|       |------>| B=2  |---     |       :       :
 	|       |       +------+   \    |       :       :       +-------+
 	+-------+       :      :    \   |       +-------+       |       |
 	                             ---------->| B->2  |------>|       |
 	                                |       +-------+       | CPU 2 |
 	                                |       | A->0  |------>|       |
 	                                |       +-------+       |       |
 	                                |       :       :       +-------+
 	                                 \      :       :
 	                                  \     +-------+
 	                                   ---->| A->1  |
 	                                        +-------+
 	                                        :       :
 If, however, a read barrier were to be placed between the load of B and the
 load of A on CPU 2:
 	CPU 1			CPU 2
 	=======================	=======================
 		{ A = 0, B = 9 }
 	STORE A=1
 	<write barrier>
 	STORE B=2
 				LOAD B
 				<read barrier>
 				LOAD A
 then the partial ordering imposed by CPU 1 will be perceived correctly by CPU
 2:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+
 	|       |------>| A=1  |------      --->| A->0  |
 	|       |       +------+      \         +-------+
 	| CPU 1 |   wwwwwwwwwwwwwwww   \    --->| B->9  |
 	|       |       +------+        |       +-------+
 	|       |------>| B=2  |---     |       :       :
 	|       |       +------+   \    |       :       :       +-------+
 	+-------+       :      :    \   |       +-------+       |       |
 	                             ---------->| B->2  |------>|       |
 	                                |       +-------+       | CPU 2 |
 	                                |       :       :       |       |
 	                                |       :       :       |       |
 	  At this point the read ---->   \  rrrrrrrrrrrrrrrrr   |       |
 	  barrier causes all effects      \     +-------+       |       |
 	  prior to the storage of B        ---->| A->1  |------>|       |
 	  to be perceptible to CPU 2            +-------+       |       |
 	                                        :       :       +-------+
 To illustrate this more completely, consider what could happen if the code
 contained a load of A either side of the read barrier:
 	CPU 1			CPU 2
 	=======================	=======================
 		{ A = 0, B = 9 }
 	STORE A=1
 	<write barrier>
 	STORE B=2
 				LOAD B
 				LOAD A [first load of A]
 				<read barrier>
 				LOAD A [second load of A]
 Even though the two loads of A both occur after the load of B, they may both
 come up with different values:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+
 	|       |------>| A=1  |------      --->| A->0  |
 	|       |       +------+      \         +-------+
 	| CPU 1 |   wwwwwwwwwwwwwwww   \    --->| B->9  |
 	|       |       +------+        |       +-------+
 	|       |------>| B=2  |---     |       :       :
 	|       |       +------+   \    |       :       :       +-------+
 	+-------+       :      :    \   |       +-------+       |       |
 	                             ---------->| B->2  |------>|       |
 	                                |       +-------+       | CPU 2 |
 	                                |       :       :       |       |
 	                                |       :       :       |       |
 	                                |       +-------+       |       |
 	                                |       | A->0  |------>| 1st   |
 	                                |       +-------+       |       |
 	  At this point the read ---->   \  rrrrrrrrrrrrrrrrr   |       |
 	  barrier causes all effects      \     +-------+       |       |
 	  prior to the storage of B        ---->| A->1  |------>| 2nd   |
 	  to be perceptible to CPU 2            +-------+       |       |
 	                                        :       :       +-------+
 But it may be that the update to A from CPU 1 becomes perceptible to CPU 2
 before the read barrier completes anyway:
 	+-------+       :      :                :       :
 	|       |       +------+                +-------+
 	|       |------>| A=1  |------      --->| A->0  |
 	|       |       +------+      \         +-------+
 	| CPU 1 |   wwwwwwwwwwwwwwww   \    --->| B->9  |
 	|       |       +------+        |       +-------+
 	|       |------>| B=2  |---     |       :       :
 	|       |       +------+   \    |       :       :       +-------+
 	+-------+       :      :    \   |       +-------+       |       |
 	                             ---------->| B->2  |------>|       |
 	                                |       +-------+       | CPU 2 |
 	                                |       :       :       |       |
 	                                 \      :       :       |       |
 	                                  \     +-------+       |       |
 	                                   ---->| A->1  |------>| 1st   |
 	                                        +-------+       |       |
 	                                    rrrrrrrrrrrrrrrrr   |       |
 	                                        +-------+       |       |
 	                                        | A->1  |------>| 2nd   |
 	                                        +-------+       |       |
 	                                        :       :       +-------+
 The guarantee is that the second load will always come up with A == 1 if the
 load of B came up with B == 2.  No such guarantee exists for the first load of
 A; that may come up with either A == 0 or A == 1.
 READ MEMORY BARRIERS VS LOAD SPECULATION
 ----------------------------------------
 Many CPUs speculate with loads: that is they see that they will need to load an
 item from memory, and they find a time where they're not using the bus for any
 other loads, and so do the load in advance - even though they haven't actually
 got to that point in the instruction execution flow yet.  This permits the
 actual load instruction to potentially complete immediately because the CPU
 already has the value to hand.
 It may turn out that the CPU didn't actually need the value - perhaps because a
 branch circumvented the load - in which case it can discard the value or just
 cache it for later use.
 Consider:
 	CPU 1	   		CPU 2
 	=======================	=======================
 	 	   		LOAD B
 	 	   		DIVIDE		} Divide instructions generally
 	 	   		DIVIDE		} take a long time to perform
 	 	   		LOAD A
 Which might appear as this:
 	                                        :       :       +-------+
 	                                        +-------+       |       |
 	                                    --->| B->2  |------>|       |
 	                                        +-------+       | CPU 2 |
 	                                        :       :DIVIDE |       |
 	                                        +-------+       |       |
 	The CPU being busy doing a --->     --->| A->0  |~~~~   |       |
 	division speculates on the              +-------+   ~   |       |
 	LOAD of A                               :       :   ~   |       |
 	                                        :       :DIVIDE |       |
 	                                        :       :   ~   |       |
 	Once the divisions are complete -->     :       :   ~-->|       |
 	the CPU can then perform the            :       :       |       |
 	LOAD with immediate effect              :       :       +-------+
 Placing a read barrier or a data dependency barrier just before the second
 load:
 	CPU 1	   		CPU 2
 	=======================	=======================
 	 	   		LOAD B
 	 	   		DIVIDE
 	 	   		DIVIDE
 				<read barrier>
 	 	   		LOAD A
 will force any value speculatively obtained to be reconsidered to an extent
 dependent on the type of barrier used.  If there was no change made to the
 speculated memory location, then the speculated value will just be used:
 	                                        :       :       +-------+
 	                                        +-------+       |       |
 	                                    --->| B->2  |------>|       |
 	                                        +-------+       | CPU 2 |
 	                                        :       :DIVIDE |       |
 	                                        +-------+       |       |
 	The CPU being busy doing a --->     --->| A->0  |~~~~   |       |
 	division speculates on the              +-------+   ~   |       |
 	LOAD of A                               :       :   ~   |       |
 	                                        :       :DIVIDE |       |
 	                                        :       :   ~   |       |
 	                                        :       :   ~   |       |
 	                                    rrrrrrrrrrrrrrrr~   |       |
 	                                        :       :   ~   |       |
 	                                        :       :   ~-->|       |
 	                                        :       :       |       |
 	                                        :       :       +-------+
 but if there was an update or an invalidation from another CPU pending, then
 the speculation will be cancelled and the value reloaded:
 	                                        :       :       +-------+
 	                                        +-------+       |       |
 	                                    --->| B->2  |------>|       |
 	                                        +-------+       | CPU 2 |
 	                                        :       :DIVIDE |       |
 	                                        +-------+       |       |
 	The CPU being busy doing a --->     --->| A->0  |~~~~   |       |
 	division speculates on the              +-------+   ~   |       |
 	LOAD of A                               :       :   ~   |       |
 	                                        :       :DIVIDE |       |
 	                                        :       :   ~   |       |
 	                                        :       :   ~   |       |
 	                                    rrrrrrrrrrrrrrrrr   |       |
 	                                        +-------+       |       |
 	The speculation is discarded --->   --->| A->1  |------>|       |
 	and an updated value is                 +-------+       |       |
 	retrieved                               :       :       +-------+
 ========================
 EXPLICIT KERNEL BARRIERS
 ========================
 The Linux kernel has a variety of different barriers that act at different
 levels:
  (*) Compiler barrier.
  (*) CPU memory barriers.
  (*) MMIO write barrier.
 COMPILER BARRIER
 ----------------
 The Linux kernel has an explicit compiler barrier function that prevents the
 compiler from moving the memory accesses either side of it to the other side:
 	barrier();
 This a general barrier - lesser varieties of compiler barrier do not exist.
 The compiler barrier has no direct effect on the CPU, which may then reorder
 things however it wishes.
 CPU MEMORY BARRIERS
 -------------------
 The Linux kernel has eight basic CPU memory barriers:
 	TYPE		MANDATORY		SMP CONDITIONAL
 	===============	=======================	===========================
 	GENERAL		mb()			smp_mb()
 	WRITE		wmb()			smp_wmb()
 	READ		rmb()			smp_rmb()
 	DATA DEPENDENCY	read_barrier_depends()	smp_read_barrier_depends()
 All CPU memory barriers unconditionally imply compiler barriers.
 SMP memory barriers are reduced to compiler barriers on uniprocessor compiled
 systems because it is assumed that a CPU will be appear to be self-consistent,
 and will order overlapping accesses correctly with respect to itself.
 [!] Note that SMP memory barriers _must_ be used to control the ordering of
 references to shared memory on SMP systems, though the use of locking instead
 is sufficient.
 Mandatory barriers should not be used to control SMP effects, since mandatory
 barriers unnecessarily impose overhead on UP systems. They may, however, be
 used to control MMIO effects on accesses through relaxed memory I/O windows.
 These are required even on non-SMP systems as they affect the order in which
 memory operations appear to a device by prohibiting both the compiler and the
 CPU from reordering them.
 There are some more advanced barrier functions:
 (*) set_mb(var, value)
     This assigns the value to the variable and then inserts at least a write
     barrier after it, depending on the function.  It isn't guaranteed to
     insert anything more than a compiler barrier in a UP compilation.
 (*) smp_mb__before_atomic_dec();
 (*) smp_mb__after_atomic_dec();
 (*) smp_mb__before_atomic_inc();
 (*) smp_mb__after_atomic_inc();
     These are for use with atomic add, subtract, increment and decrement
     functions that don't return a value, especially when used for reference
     counting.  These functions do not imply memory barriers.
     As an example, consider a piece of code that marks an object as being dead
     and then decrements the object's reference count:
 	obj->dead = 1;
 	smp_mb__before_atomic_dec();
 	atomic_dec(&obj->ref_count);
     This makes sure that the death mark on the object is perceived to be set
     *before* the reference counter is decremented.
     See Documentation/atomic_ops.txt for more information.  See the "Atomic
     operations" subsection for information on where to use these.
 (*) smp_mb__before_clear_bit(void);
 (*) smp_mb__after_clear_bit(void);
     These are for use similar to the atomic inc/dec barriers.  These are
     typically used for bitwise unlocking operations, so care must be taken as
     there are no implicit memory barriers here either.
     Consider implementing an unlock operation of some nature by clearing a
     locking bit.  The clear_bit() would then need to be barriered like this:
 	smp_mb__before_clear_bit();
 	clear_bit( ... );
     This prevents memory operations before the clear leaking to after it.  See
     the subsection on "Locking Functions" with reference to UNLOCK operation
     implications.
     See Documentation/atomic_ops.txt for more information.  See the "Atomic
     operations" subsection for information on where to use these.
 MMIO WRITE BARRIER
 ------------------
 The Linux kernel also has a special barrier for use with memory-mapped I/O
 writes:
 	mmiowb();
 This is a variation on the mandatory write barrier that causes writes to weakly
 ordered I/O regions to be partially ordered.  Its effects may go beyond the
 CPU->Hardware interface and actually affect the hardware at some level.
 See the subsection "Locks vs I/O accesses" for more information.
 ===============================
 IMPLICIT KERNEL MEMORY BARRIERS
 ===============================
 Some of the other functions in the linux kernel imply memory barriers, amongst
 which are locking and scheduling functions.
 This specification is a _minimum_ guarantee; any particular architecture may
 provide more substantial guarantees, but these may not be relied upon outside
 of arch specific code.
 LOCKING FUNCTIONS
 -----------------
 The Linux kernel has a number of locking constructs:
 (*) spin locks
 (*) R/W spin locks
 (*) mutexes
 (*) semaphores
 (*) R/W semaphores
 (*) RCU
 In all cases there are variants on "LOCK" operations and "UNLOCK" operations
 for each construct.  These operations all imply certain barriers:
 (1) LOCK operation implication:
     Memory operations issued after the LOCK will be completed after the LOCK
     operation has completed.
     Memory operations issued before the LOCK may be completed after the LOCK
     operation has completed.
 (2) UNLOCK operation implication:
     Memory operations issued before the UNLOCK will be completed before the
     UNLOCK operation has completed.
     Memory operations issued after the UNLOCK may be completed before the
     UNLOCK operation has completed.
 (3) LOCK vs LOCK implication:
     All LOCK operations issued before another LOCK operation will be completed
     before that LOCK operation.
 (4) LOCK vs UNLOCK implication:
     All LOCK operations issued before an UNLOCK operation will be completed
     before the UNLOCK operation.
     All UNLOCK operations issued before a LOCK operation will be completed
     before the LOCK operation.
 (5) Failed conditional LOCK implication:
     Certain variants of the LOCK operation may fail, either due to being
     unable to get the lock immediately, or due to receiving an unblocked
     signal whilst asleep waiting for the lock to become available.  Failed
     locks do not imply any sort of barrier.
 Therefore, from (1), (2) and (4) an UNLOCK followed by an unconditional LOCK is
 equivalent to a full barrier, but a LOCK followed by an UNLOCK is not.
 [!] Note: one of the consequence of LOCKs and UNLOCKs being only one-way
    barriers is that the effects instructions outside of a critical section may
    seep into the inside of the critical section.
 A LOCK followed by an UNLOCK may not be assumed to be full memory barrier
 because it is possible for an access preceding the LOCK to happen after the
 LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the
 two accesses can themselves then cross:
 	*A = a;
 	LOCK
 	UNLOCK
 	*B = b;
 may occur as:
 	LOCK, STORE *B, STORE *A, UNLOCK
 Locks and semaphores may not provide any guarantee of ordering on UP compiled
 systems, and so cannot be counted on in such a situation to actually achieve
 anything at all - especially with respect to I/O accesses - unless combined
 with interrupt disabling operations.
 See also the section on "Inter-CPU locking barrier effects".
 As an example, consider the following:
 	*A = a;
 	*B = b;
 	LOCK
 	*C = c;
 	*D = d;
 	UNLOCK
 	*E = e;
 	*F = f;
 The following sequence of events is acceptable:
 	LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK
 	[+] Note that {*F,*A} indicates a combined access.
 But none of the following are:
 	{*F,*A}, *B,	LOCK, *C, *D,	UNLOCK, *E
 	*A, *B, *C,	LOCK, *D,	UNLOCK, *E, *F
 	*A, *B,		LOCK, *C,	UNLOCK, *D, *E, *F
 	*B,		LOCK, *C, *D,	UNLOCK, {*F,*A}, *E
 INTERRUPT DISABLING FUNCTIONS
 -----------------------------
 Functions that disable interrupts (LOCK equivalent) and enable interrupts
 (UNLOCK equivalent) will act as compiler barriers only.  So if memory or I/O
 barriers are required in such a situation, they must be provided from some
 other means.
 MISCELLANEOUS FUNCTIONS
 -----------------------
 Other functions that imply barriers:
 (*) schedule() and similar imply full memory barriers.
 =================================
 INTER-CPU LOCKING BARRIER EFFECTS
 =================================
 On SMP systems locking primitives give a more substantial form of barrier: one
 that does affect memory access ordering on other CPUs, within the context of
 conflict on any particular lock.
 LOCKS VS MEMORY ACCESSES
 ------------------------
 Consider the following: the system has a pair of spinlocks (M) and (Q), and
 three CPUs; then should the following sequence of events occur:
 	CPU 1				CPU 2
 	===============================	===============================
 	*A = a;				*E = e;
 	LOCK M				LOCK Q
 	*B = b;				*F = f;
 	*C = c;				*G = g;
 	UNLOCK M			UNLOCK Q
 	*D = d;				*H = h;
 Then there is no guarantee as to what order CPU #3 will see the accesses to *A
 through *H occur in, other than the constraints imposed by the separate locks
 on the separate CPUs. It might, for example, see:
 	*E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M
 But it won't see any of:
 	*B, *C or *D preceding LOCK M
 	*A, *B or *C following UNLOCK M
 	*F, *G or *H preceding LOCK Q
 	*E, *F or *G following UNLOCK Q
 However, if the following occurs:
 	CPU 1				CPU 2
 	===============================	===============================
 	*A = a;
 	LOCK M		[1]
 	*B = b;
 	*C = c;
 	UNLOCK M	[1]
 	*D = d;				*E = e;
 					LOCK M		[2]
 					*F = f;
 					*G = g;
 					UNLOCK M	[2]
 					*H = h;
 CPU #3 might see:
 	*E, LOCK M [1], *C, *B, *A, UNLOCK M [1],
 		LOCK M [2], *H, *F, *G, UNLOCK M [2], *D
 But assuming CPU #1 gets the lock first, it won't see any of:
 	*B, *C, *D, *F, *G or *H preceding LOCK M [1]
 	*A, *B or *C following UNLOCK M [1]
 	*F, *G or *H preceding LOCK M [2]
 	*A, *B, *C, *E, *F or *G following UNLOCK M [2]
 LOCKS VS I/O ACCESSES
 ---------------------
 Under certain circumstances (especially involving NUMA), I/O accesses within
 two spinlocked sections on two different CPUs may be seen as interleaved by the
 PCI bridge, because the PCI bridge does not necessarily participate in the
 cache-coherence protocol, and is therefore incapable of issuing the required
 read memory barriers.
 For example:
 	CPU 1				CPU 2
 	===============================	===============================
 	spin_lock(Q)
 	writel(0, ADDR)
 	writel(1, DATA);
 	spin_unlock(Q);
 					spin_lock(Q);
 					writel(4, ADDR);
 					writel(5, DATA);
 					spin_unlock(Q);
 may be seen by the PCI bridge as follows:
 	STORE *ADDR = 0, STORE *ADDR = 4, STORE *DATA = 1, STORE *DATA = 5
 which would probably cause the hardware to malfunction.
 What is necessary here is to intervene with an mmiowb() before dropping the
 spinlock, for example:
 	CPU 1				CPU 2
 	===============================	===============================
 	spin_lock(Q)
 	writel(0, ADDR)
 	writel(1, DATA);
 	mmiowb();
 	spin_unlock(Q);
 					spin_lock(Q);
 					writel(4, ADDR);
 					writel(5, DATA);
 					mmiowb();
 					spin_unlock(Q);
 this will ensure that the two stores issued on CPU #1 appear at the PCI bridge
 before either of the stores issued on CPU #2.
 Furthermore, following a store by a load to the same device obviates the need
 for an mmiowb(), because the load forces the store to complete before the load
 is performed:
 	CPU 1				CPU 2
 	===============================	===============================
 	spin_lock(Q)
 	writel(0, ADDR)
 	a = readl(DATA);
 	spin_unlock(Q);
 					spin_lock(Q);
 					writel(4, ADDR);
 					b = readl(DATA);
 					spin_unlock(Q);
 See Documentation/DocBook/deviceiobook.tmpl for more information.
 =================================
 WHERE ARE MEMORY BARRIERS NEEDED?
 =================================
 Under normal operation, memory operation reordering is generally not going to
 be a problem as a single-threaded linear piece of code will still appear to
 work correctly, even if it's in an SMP kernel.  There are, however, three
 circumstances in which reordering definitely _could_ be a problem:
 (*) Interprocessor interaction.
 (*) Atomic operations.
 (*) Accessing devices (I/O).
 (*) Interrupts.
 INTERPROCESSOR INTERACTION
 --------------------------
 When there's a system with more than one processor, more than one CPU in the
 system may be working on the same data set at the same time.  This can cause
 synchronisation problems, and the usual way of dealing with them is to use
 locks.  Locks, however, are quite expensive, and so it may be preferable to
 operate without the use of a lock if at all possible.  In such a case
 operations that affect both CPUs may have to be carefully ordered to prevent
 a malfunction.
 Consider, for example, the R/W semaphore slow path.  Here a waiting process is
 queued on the semaphore, by virtue of it having a piece of its stack linked to
 the semaphore's list of waiting processes:
 	struct rw_semaphore {
 		...
 		spinlock_t lock;
 		struct list_head waiters;
 	};
 	struct rwsem_waiter {
 		struct list_head list;
 		struct task_struct *task;
 	};
 To wake up a particular waiter, the up_read() or up_write() functions have to:
 (1) read the next pointer from this waiter's record to know as to where the
     next waiter record is;
 (4) read the pointer to the waiter's task structure;
 (3) clear the task pointer to tell the waiter it has been given the semaphore;
 (4) call wake_up_process() on the task; and
 (5) release the reference held on the waiter's task struct.
 In otherwords, it has to perform this sequence of events:
 	LOAD waiter->list.next;
 	LOAD waiter->task;
 	STORE waiter->task;
 	CALL wakeup
 	RELEASE task
 and if any of these steps occur out of order, then the whole thing may
 malfunction.
 Once it has queued itself and dropped the semaphore lock, the waiter does not
 get the lock again; it instead just waits for its task pointer to be cleared
 before proceeding.  Since the record is on the waiter's stack, this means that
 if the task pointer is cleared _before_ the next pointer in the list is read,
 another CPU might start processing the waiter and might clobber the waiter's
 stack before the up*() function has a chance to read the next pointer.
 Consider then what might happen to the above sequence of events:
 	CPU 1				CPU 2
 	===============================	===============================
 					down_xxx()
 					Queue waiter
 					Sleep
 	up_yyy()
 	LOAD waiter->task;
 	STORE waiter->task;
 					Woken up by other event
 	<preempt>
 					Resume processing
 					down_xxx() returns
 					call foo()
 					foo() clobbers *waiter
 	</preempt>
 	LOAD waiter->list.next;
 	--- OOPS ---
 This could be dealt with using the semaphore lock, but then the down_xxx()
 function has to needlessly get the spinlock again after being woken up.
 The way to deal with this is to insert a general SMP memory barrier:
 	LOAD waiter->list.next;
 	LOAD waiter->task;
 	smp_mb();
 	STORE waiter->task;
 	CALL wakeup
 	RELEASE task
 In this case, the barrier makes a guarantee that all memory accesses before the
 barrier will appear to happen before all the memory accesses after the barrier
 with respect to the other CPUs on the system.  It does _not_ guarantee that all
 the memory accesses before the barrier will be complete by the time the barrier
 instruction itself is complete.
 On a UP system - where this wouldn't be a problem - the smp_mb() is just a
 compiler barrier, thus making sure the compiler emits the instructions in the
 right order without actually intervening in the CPU.  Since there's only one
 CPU, that CPU's dependency ordering logic will take care of everything else.
 ATOMIC OPERATIONS
 -----------------
 Whilst they are technically interprocessor interaction considerations, atomic
 operations are noted specially as some of them imply full memory barriers and
 some don't, but they're very heavily relied on as a group throughout the
 kernel.
 Any atomic operation that modifies some state in memory and returns information
 about the state (old or new) implies an SMP-conditional general memory barrier
 (smp_mb()) on each side of the actual operation.  These include:
 	xchg();
 	cmpxchg();
 	atomic_cmpxchg();
 	atomic_inc_return();
 	atomic_dec_return();
 	atomic_add_return();
 	atomic_sub_return();
 	atomic_inc_and_test();
 	atomic_dec_and_test();
 	atomic_sub_and_test();
 	atomic_add_negative();
 	atomic_add_unless();
 	test_and_set_bit();
 	test_and_clear_bit();
 	test_and_change_bit();
 These are used for such things as implementing LOCK-class and UNLOCK-class
 operations and adjusting reference counters towards object destruction, and as
 such the implicit memory barrier effects are necessary.
 The following operation are potential problems as they do _not_ imply memory
 barriers, but might be used for implementing such things as UNLOCK-class
 operations:
 	atomic_set();
 	set_bit();
 	clear_bit();
 	change_bit();
 With these the appropriate explicit memory barrier should be used if necessary
 (smp_mb__before_clear_bit() for instance).
 The following also do _not_ imply memory barriers, and so may require explicit
 memory barriers under some circumstances (smp_mb__before_atomic_dec() for
 instance)):
 	atomic_add();
 	atomic_sub();
 	atomic_inc();
 	atomic_dec();
 If they're used for statistics generation, then they probably don't need memory
 barriers, unless there's a coupling between statistical data.
 If they're used for reference counting on an object to control its lifetime,
 they probably don't need memory barriers because either the reference count
 will be adjusted inside a locked section, or the caller will already hold
 sufficient references to make the lock, and thus a memory barrier unnecessary.
 If they're used for constructing a lock of some description, then they probably
 do need memory barriers as a lock primitive generally has to do things in a
 specific order.
 Basically, each usage case has to be carefully considered as to whether memory
 barriers are needed or not.
 [!] Note that special memory barrier primitives are available for these
 situations because on some CPUs the atomic instructions used imply full memory
 barriers, and so barrier instructions are superfluous in conjunction with them,
 and in such cases the special barrier primitives will be no-ops.
 See Documentation/atomic_ops.txt for more information.
 ACCESSING DEVICES
 -----------------
 Many devices can be memory mapped, and so appear to the CPU as if they're just
 a set of memory locations.  To control such a device, the driver usually has to
 make the right memory accesses in exactly the right order.
 However, having a clever CPU or a clever compiler creates a potential problem
 in that the carefully sequenced accesses in the driver code won't reach the
 device in the requisite order if the CPU or the compiler thinks it is more
 efficient to reorder, combine or merge accesses - something that would cause
 the device to malfunction.
 Inside of the Linux kernel, I/O should be done through the appropriate accessor
 routines - such as inb() or writel() - which know how to make such accesses
 appropriately sequential.  Whilst this, for the most part, renders the explicit
 use of memory barriers unnecessary, there are a couple of situations where they
 might be needed:
 (1) On some systems, I/O stores are not strongly ordered across all CPUs, and
     so for _all_ general drivers locks should be used and mmiowb() must be
     issued prior to unlocking the critical section.
 (2) If the accessor functions are used to refer to an I/O memory window with
     relaxed memory access properties, then _mandatory_ memory barriers are
     required to enforce ordering.
 See Documentation/DocBook/deviceiobook.tmpl for more information.
 INTERRUPTS
 ----------
 A driver may be interrupted by its own interrupt service routine, and thus the
 two parts of the driver may interfere with each other's attempts to control or
 access the device.
 This may be alleviated - at least in part - by disabling local interrupts (a
 form of locking), such that the critical operations are all contained within
 the interrupt-disabled section in the driver.  Whilst the driver's interrupt
 routine is executing, the driver's core may not run on the same CPU, and its
 interrupt is not permitted to happen again until the current interrupt has been
 handled, thus the interrupt handler does not need to lock against that.
 However, consider a driver that was talking to an ethernet card that sports an
 address register and a data register.  If that driver's core talks to the card
 under interrupt-disablement and then the driver's interrupt handler is invoked:
 	LOCAL IRQ DISABLE
 	writew(ADDR, 3);
 	writew(DATA, y);
 	LOCAL IRQ ENABLE
 	<interrupt>
 	writew(ADDR, 4);
 	q = readw(DATA);
 	</interrupt>
 The store to the data register might happen after the second store to the
 address register if ordering rules are sufficiently relaxed:
 	STORE *ADDR = 3, STORE *ADDR = 4, STORE *DATA = y, q = LOAD *DATA
 If ordering rules are relaxed, it must be assumed that accesses done inside an
 interrupt disabled section may leak outside of it and may interleave with
 accesses performed in an interrupt - and vice versa - unless implicit or
 explicit barriers are used.
 Normally this won't be a problem because the I/O accesses done inside such
 sections will include synchronous load operations on strictly ordered I/O
 registers that form implicit I/O barriers. If this isn't sufficient then an
 mmiowb() may need to be used explicitly.
 A similar situation may occur between an interrupt routine and two routines
 running on separate CPUs that communicate with each other. If such a case is
 likely, then interrupt-disabling locks should be used to guarantee ordering.
 ==========================
 KERNEL I/O BARRIER EFFECTS
 ==========================
 When accessing I/O memory, drivers should use the appropriate accessor
 functions:
 (*) inX(), outX():
     These are intended to talk to I/O space rather than memory space, but
     that's primarily a CPU-specific concept. The i386 and x86_64 processors do
     indeed have special I/O space access cycles and instructions, but many
     CPUs don't have such a concept.
     The PCI bus, amongst others, defines an I/O space concept - which on such
     CPUs as i386 and x86_64 cpus readily maps to the CPU's concept of I/O
     space.  However, it may also be mapped as a virtual I/O space in the CPU's
     memory map, particularly on those CPUs that don't support alternate I/O
     spaces.
     Accesses to this space may be fully synchronous (as on i386), but
     intermediary bridges (such as the PCI host bridge) may not fully honour
     that.
     They are guaranteed to be fully ordered with respect to each other.
     They are not guaranteed to be fully ordered with respect to other types of
     memory and I/O operation.
 (*) readX(), writeX():
     Whether these are guaranteed to be fully ordered and uncombined with
     respect to each other on the issuing CPU depends on the characteristics
     defined for the memory window through which they're accessing. On later
     i386 architecture machines, for example, this is controlled by way of the
     MTRR registers.
     Ordinarily, these will be guaranteed to be fully ordered and uncombined,,
     provided they're not accessing a prefetchable device.
     However, intermediary hardware (such as a PCI bridge) may indulge in
     deferral if it so wishes; to flush a store, a load from the same location
     is preferred[*], but a load from the same device or from configuration
     space should suffice for PCI.
     [*] NOTE! attempting to load from the same location as was written to may
     	 cause a malfunction - consider the 16550 Rx/Tx serial registers for
     	 example.
     Used with prefetchable I/O memory, an mmiowb() barrier may be required to
     force stores to be ordered.
     Please refer to the PCI specification for more information on interactions
     between PCI transactions.
 (*) readX_relaxed()
     These are similar to readX(), but are not guaranteed to be ordered in any
     way. Be aware that there is no I/O read barrier available.
 (*) ioreadX(), iowriteX()
     These will perform as appropriate for the type of access they're actually
     doing, be it inX()/outX() or readX()/writeX().
 ========================================
 ASSUMED MINIMUM EXECUTION ORDERING MODEL
 ========================================
 It has to be assumed that the conceptual CPU is weakly-ordered but that it will
 maintain the appearance of program causality with respect to itself.  Some CPUs
 (such as i386 or x86_64) are more constrained than others (such as powerpc or
 frv), and so the most relaxed case (namely DEC Alpha) must be assumed outside
 of arch-specific code.
 This means that it must be considered that the CPU will execute its instruction
 stream in any order it feels like - or even in parallel - provided that if an
 instruction in the stream depends on the an earlier instruction, then that
 earlier instruction must be sufficiently complete[*] before the later
 instruction may proceed; in other words: provided that the appearance of
 causality is maintained.
 [*] Some instructions have more than one effect - such as changing the
     condition codes, changing registers or changing memory - and different
     instructions may depend on different effects.
 A CPU may also discard any instruction sequence that winds up having no
 ultimate effect.  For example, if two adjacent instructions both load an
 immediate value into the same register, the first may be discarded.
 Similarly, it has to be assumed that compiler might reorder the instruction
 stream in any way it sees fit, again provided the appearance of causality is
 maintained.
 ============================
 THE EFFECTS OF THE CPU CACHE
 ============================
 The way cached memory operations are perceived across the system is affected to
 a certain extent by the caches that lie between CPUs and memory, and by the
 memory coherence system that maintains the consistency of state in the system.
 As far as the way a CPU interacts with another part of the system through the
 caches goes, the memory system has to include the CPU's caches, and memory
 barriers for the most part act at the interface between the CPU and its cache
 (memory barriers logically act on the dotted line in the following diagram):
 	    <--- CPU --->         :       <----------- Memory ----------->
 	                          :
 	+--------+    +--------+  :   +--------+    +-----------+
 	|        |    |        |  :   |        |    |           |    +--------+
 	|  CPU   |    | Memory |  :   | CPU    |    |           |    |	      |
 	|  Core  |--->| Access |----->| Cache  |<-->|           |    |	      |
 	|        |    | Queue  |  :   |        |    |           |--->| Memory |
 	|        |    |        |  :   |        |    |           |    |	      |
 	+--------+    +--------+  :   +--------+    |           |    | 	      |
 	                          :                 | Cache     |    +--------+
 	                          :                 | Coherency |
 	                          :                 | Mechanism |    +--------+
 	+--------+    +--------+  :   +--------+    |           |    |	      |
 	|        |    |        |  :   |        |    |           |    |        |
 	|  CPU   |    | Memory |  :   | CPU    |    |           |--->| Device |
 	|  Core  |--->| Access |----->| Cache  |<-->|           |    | 	      |
 	|        |    | Queue  |  :   |        |    |           |    | 	      |
 	|        |    |        |  :   |        |    |           |    +--------+
 	+--------+    +--------+  :   +--------+    +-----------+
 	                          :
 	                          :
 Although any particular load or store may not actually appear outside of the
 CPU that issued it since it may have been satisfied within the CPU's own cache,
 it will still appear as if the full memory access had taken place as far as the
 other CPUs are concerned since the cache coherency mechanisms will migrate the
 cacheline over to the accessing CPU and propagate the effects upon conflict.
 The CPU core may execute instructions in any order it deems fit, provided the
 expected program causality appears to be maintained.  Some of the instructions
 generate load and store operations which then go into the queue of memory
 accesses to be performed.  The core may place these in the queue in any order
 it wishes, and continue execution until it is forced to wait for an instruction
 to complete.
 What memory barriers are concerned with is controlling the order in which
 accesses cross from the CPU side of things to the memory side of things, and
 the order in which the effects are perceived to happen by the other observers
 in the system.
 [!] Memory barriers are _not_ needed within a given CPU, as CPUs always see
 their own loads and stores as if they had happened in program order.
 [!] MMIO or other device accesses may bypass the cache system.  This depends on
 the properties of the memory window through which devices are accessed and/or
 the use of any special device communication instructions the CPU may have.
 CACHE COHERENCY
 ---------------
 Life isn't quite as simple as it may appear above, however: for while the
 caches are expected to be coherent, there's no guarantee that that coherency
 will be ordered.  This means that whilst changes made on one CPU will
 eventually become visible on all CPUs, there's no guarantee that they will
 become apparent in the same order on those other CPUs.
 Consider dealing with a system that has pair of CPUs (1 & 2), each of which has
 a pair of parallel data caches (CPU 1 has A/B, and CPU 2 has C/D):
 	            :
 	            :                          +--------+
 	            :      +---------+         |        |
 	+--------+  : +--->| Cache A |<------->|        |
 	|        |  : |    +---------+         |        |
 	|  CPU 1 |<---+                        |        |
 	|        |  : |    +---------+         |        |
 	+--------+  : +--->| Cache B |<------->|        |
 	            :      +---------+         |        |
 	            :                          | Memory |
 	            :      +---------+         | System |
 	+--------+  : +--->| Cache C |<------->|        |
 	|        |  : |    +---------+         |        |
 	|  CPU 2 |<---+                        |        |
 	|        |  : |    +---------+         |        |
 	+--------+  : +--->| Cache D |<------->|        |
 	            :      +---------+         |        |
 	            :                          +--------+
 	            :
 Imagine the system has the following properties:
 (*) an odd-numbered cache line may be in cache A, cache C or it may still be
     resident in memory;
 (*) an even-numbered cache line may be in cache B, cache D or it may still be
     resident in memory;
 (*) whilst the CPU core is interrogating one cache, the other cache may be
     making use of the bus to access the rest of the system - perhaps to
     displace a dirty cacheline or to do a speculative load;
 (*) each cache has a queue of operations that need to be applied to that cache
     to maintain coherency with the rest of the system;
 (*) the coherency queue is not flushed by normal loads to lines already
     present in the cache, even though the contents of the queue may
     potentially effect those loads.
 Imagine, then, that two writes are made on the first CPU, with a write barrier
 between them to guarantee that they will appear to reach that CPU's caches in
 the requisite order:
 	CPU 1		CPU 2		COMMENT
 	===============	===============	=======================================
 					u == 0, v == 1 and p == &u, q == &u
 	v = 2;
 	smp_wmb();			Make sure change to v visible before
 					 change to p
 	<A:modify v=2>			v is now in cache A exclusively
 	p = &v;
 	<B:modify p=&v>			p is now in cache B exclusively
 The write memory barrier forces the other CPUs in the system to perceive that
 the local CPU's caches have apparently been updated in the correct order.  But
 now imagine that the second CPU that wants to read those values:
 	CPU 1		CPU 2		COMMENT
 	===============	===============	=======================================
 	...
 			q = p;
 			x = *q;
 The above pair of reads may then fail to happen in expected order, as the
 cacheline holding p may get updated in one of the second CPU's caches whilst
 the update to the cacheline holding v is delayed in the other of the second
 CPU's caches by some other cache event:
 	CPU 1		CPU 2		COMMENT
 	===============	===============	=======================================
 					u == 0, v == 1 and p == &u, q == &u
 	v = 2;
 	smp_wmb();
 	<A:modify v=2>	<C:busy>
 			<C:queue v=2>
 	p = &v;		q = p;
 			<D:request p>
 	<B:modify p=&v>	<D:commit p=&v>
 		  	<D:read p>
 			x = *q;
 			<C:read *q>	Reads from v before v updated in cache
 			<C:unbusy>
 			<C:commit v=2>
 Basically, whilst both cachelines will be updated on CPU 2 eventually, there's
 no guarantee that, without intervention, the order of update will be the same
 as that committed on CPU 1.
 To intervene, we need to interpolate a data dependency barrier or a read
 barrier between the loads.  This will force the cache to commit its coherency
 queue before processing any further requests:
 	CPU 1		CPU 2		COMMENT
 	===============	===============	=======================================
 					u == 0, v == 1 and p == &u, q == &u
 	v = 2;
 	smp_wmb();
 	<A:modify v=2>	<C:busy>
 			<C:queue v=2>
 	p = &b;		q = p;
 			<D:request p>
 	<B:modify p=&v>	<D:commit p=&v>
 		  	<D:read p>
 			smp_read_barrier_depends()
 			<C:unbusy>
 			<C:commit v=2>
 			x = *q;
 			<C:read *q>	Reads from v after v updated in cache
 This sort of problem can be encountered on DEC Alpha processors as they have a
 split cache that improves performance by making better use of the data bus.
 Whilst most CPUs do imply a data dependency barrier on the read when a memory
 access depends on a read, not all do, so it may not be relied on.
 Other CPUs may also have split caches, but must coordinate between the various
 cachelets for normal memory accesses.  The semantics of the Alpha removes the
 need for coordination in absence of memory barriers.
 CACHE COHERENCY VS DMA
 ----------------------
 Not all systems maintain cache coherency with respect to devices doing DMA.  In
 such cases, a device attempting DMA may obtain stale data from RAM because
 dirty cache lines may be resident in the caches of various CPUs, and may not
 have been written back to RAM yet.  To deal with this, the appropriate part of
 the kernel must flush the overlapping bits of cache on each CPU (and maybe
 invalidate them as well).
 In addition, the data DMA'd to RAM by a device may be overwritten by dirty
 cache lines being written back to RAM from a CPU's cache after the device has
 installed its own data, or cache lines simply present in a CPUs cache may
 simply obscure the fact that RAM has been updated, until at such time as the
 cacheline is discarded from the CPU's cache and reloaded.  To deal with this,
 the appropriate part of the kernel must invalidate the overlapping bits of the
 cache on each CPU.
 See Documentation/cachetlb.txt for more information on cache management.
 CACHE COHERENCY VS MMIO
 -----------------------
 Memory mapped I/O usually takes place through memory locations that are part of
 a window in the CPU's memory space that have different properties assigned than
 the usual RAM directed window.
 Amongst these properties is usually the fact that such accesses bypass the
 caching entirely and go directly to the device buses.  This means MMIO accesses
 may, in effect, overtake accesses to cached memory that were emitted earlier.
 A memory barrier isn't sufficient in such a case, but rather the cache must be
 flushed between the cached memory write and the MMIO access if the two are in
 any way dependent.
 =========================
 THE THINGS CPUS GET UP TO
 =========================
 A programmer might take it for granted that the CPU will perform memory
 operations in exactly the order specified, so that if a CPU is, for example,
 given the following piece of code to execute:
 	a = *A;
 	*B = b;
 	c = *C;
 	d = *D;
 	*E = e;
 They would then expect that the CPU will complete the memory operation for each
 instruction before moving on to the next one, leading to a definite sequence of
 operations as seen by external observers in the system:
 	LOAD *A, STORE *B, LOAD *C, LOAD *D, STORE *E.
 Reality is, of course, much messier.  With many CPUs and compilers, the above
 assumption doesn't hold because:
 (*) loads are more likely to need to be completed immediately to permit
     execution progress, whereas stores can often be deferred without a
     problem;
 (*) loads may be done speculatively, and the result discarded should it prove
     to have been unnecessary;
 (*) loads may be done speculatively, leading to the result having being
     fetched at the wrong time in the expected sequence of events;
 (*) the order of the memory accesses may be rearranged to promote better use
     of the CPU buses and caches;
 (*) loads and stores may be combined to improve performance when talking to
     memory or I/O hardware that can do batched accesses of adjacent locations,
     thus cutting down on transaction setup costs (memory and PCI devices may
     both be able to do this); and
 (*) the CPU's data cache may affect the ordering, and whilst cache-coherency
     mechanisms may alleviate this - once the store has actually hit the cache
     - there's no guarantee that the coherency management will be propagated in
     order to other CPUs.
 So what another CPU, say, might actually observe from the above piece of code
 is:
 	LOAD *A, ..., LOAD {*C,*D}, STORE *E, STORE *B
 	(Where "LOAD {*C,*D}" is a combined load)
 However, it is guaranteed that a CPU will be self-consistent: it will see its
 _own_ accesses appear to be correctly ordered, without the need for a memory
 barrier.  For instance with the following code:
 	U = *A;
 	*A = V;
 	*A = W;
 	X = *A;
 	*A = Y;
 	Z = *A;
 and assuming no intervention by an external influence, it can be assumed that
 the final result will appear to be:
 	U == the original value of *A
 	X == W
 	Z == Y
 	*A == Y
 The code above may cause the CPU to generate the full sequence of memory
 accesses:
 	U=LOAD *A, STORE *A=V, STORE *A=W, X=LOAD *A, STORE *A=Y, Z=LOAD *A
 in that order, but, without intervention, the sequence may have almost any
 combination of elements combined or discarded, provided the program's view of
 the world remains consistent.
 The compiler may also combine, discard or defer elements of the sequence before
 the CPU even sees them.
 For instance:
 	*A = V;
 	*A = W;
 may be reduced to:
 	*A = W;
 since, without a write barrier, it can be assumed that the effect of the
 storage of V to *A is lost.  Similarly:
 	*A = Y;
 	Z = *A;
 may, without a memory barrier, be reduced to:
 	*A = Y;
 	Z = Y;
 and the LOAD operation never appear outside of the CPU.
 AND THEN THERE'S THE ALPHA
 --------------------------
 The DEC Alpha CPU is one of the most relaxed CPUs there is.  Not only that,
 some versions of the Alpha CPU have a split data cache, permitting them to have
 two semantically related cache lines updating at separate times.  This is where
 the data dependency barrier really becomes necessary as this synchronises both
 caches with the memory coherence system, thus making it seem like pointer
 changes vs new data occur in the right order.
 The Alpha defines the Linux's kernel's memory barrier model.
 See the subsection on "Cache Coherency" above.
 ==========
 REFERENCES
 ==========
 Alpha AXP Architecture Reference Manual, Second Edition (Sites & Witek,
 Digital Press)
 	Chapter 5.2: Physical Address Space Characteristics
 	Chapter 5.4: Caches and Write Buffers
 	Chapter 5.5: Data Sharing
 	Chapter 5.6: Read/Write Ordering
 AMD64 Architecture Programmer's Manual Volume 2: System Programming
 	Chapter 7.1: Memory-Access Ordering
 	Chapter 7.4: Buffering and Combining Memory Writes
 IA-32 Intel Architecture Software Developer's Manual, Volume 3:
 System Programming Guide
 	Chapter 7.1: Locked Atomic Operations
 	Chapter 7.2: Memory Ordering
 	Chapter 7.4: Serializing Instructions
 The SPARC Architecture Manual, Version 9
 	Chapter 8: Memory Models
 	Appendix D: Formal Specification of the Memory Models
 	Appendix J: Programming with the Memory Models
 UltraSPARC Programmer Reference Manual
 	Chapter 5: Memory Accesses and Cacheability
 	Chapter 15: Sparc-V9 Memory Models
 UltraSPARC III Cu User's Manual
 	Chapter 9: Memory Models
 UltraSPARC IIIi Processor User's Manual
 	Chapter 8: Memory Models
 UltraSPARC Architecture 2005
 	Chapter 9: Memory
 	Appendix D: Formal Specifications of the Memory Models
 UltraSPARC T1 Supplement to the UltraSPARC Architecture 2005
 	Chapter 8: Memory Models
 	Appendix F: Caches and Cache Coherency
 Solaris Internals, Core Kernel Architecture, p63-68:
 	Chapter 3.3: Hardware Considerations for Locks and
 			Synchronization
 Unix Systems for Modern Architectures, Symmetric Multiprocessing and Caching
 for Kernel Programmers:
 	Chapter 13: Other Memory Models
 Intel Itanium Architecture Software Developer's Manual: Volume 1:
 	Section 2.6: Speculation
 	Section 4.4: Memory Access
 		Linux Ethernet Bonding Driver HOWTO
 		Latest update: 24 April 2006
 Initial release : Thomas Davis <tadavis at lbl.gov>
 Corrections, HA extensions : 2000/10/03-15 :
  - Willy Tarreau <willy at meta-x.org>
  - Constantine Gavrilov <const-g at xpert.com>
  - Chad N. Tindel <ctindel at ieee dot org>
  - Janice Girouard <girouard at us dot ibm dot com>
  - Jay Vosburgh <fubar at us dot ibm dot com>
 Reorganized and updated Feb 2005 by Jay Vosburgh
 Added Sysfs information: 2006/04/24
  - Mitch Williams <mitch.a.williams at intel.com>
 Introduction
 ============
 	The Linux bonding driver provides a method for aggregating
 multiple network interfaces into a single logical "bonded" interface.
 The behavior of the bonded interfaces depends upon the mode; generally
 speaking, modes provide either hot standby or load balancing services.
 Additionally, link integrity monitoring may be performed.
 	The bonding driver originally came from Donald Becker's
 beowulf patches for kernel 2.0. It has changed quite a bit since, and
 the original tools from extreme-linux and beowulf sites will not work
 with this version of the driver.
 	For new versions of the driver, updated userspace tools, and
 who to ask for help, please follow the links at the end of this file.
 Table of Contents
 =================
 1. Bonding Driver Installation
 2. Bonding Driver Options
 3. Configuring Bonding Devices
 3.1	Configuration with Sysconfig Support
 3.1.1		Using DHCP with Sysconfig
 3.1.2		Configuring Multiple Bonds with Sysconfig
 3.2	Configuration with Initscripts Support
 3.2.1		Using DHCP with Initscripts
 3.2.2		Configuring Multiple Bonds with Initscripts
 3.3	Configuring Bonding Manually with Ifenslave
 3.3.1		Configuring Multiple Bonds Manually
 3.4	Configuring Bonding Manually via Sysfs
 4. Querying Bonding Configuration
 4.1	Bonding Configuration
 4.2	Network Configuration
 5. Switch Configuration
 6. 802.1q VLAN Support
 7. Link Monitoring
 7.1	ARP Monitor Operation
 7.2	Configuring Multiple ARP Targets
 7.3	MII Monitor Operation
 8. Potential Trouble Sources
 8.1	Adventures in Routing
 8.2	Ethernet Device Renaming
 8.3	Painfully Slow Or No Failed Link Detection By Miimon
 9. SNMP agents
 10. Promiscuous mode
 11. Configuring Bonding for High Availability
 11.1	High Availability in a Single Switch Topology
 11.2	High Availability in a Multiple Switch Topology
 11.2.1		HA Bonding Mode Selection for Multiple Switch Topology
 11.2.2		HA Link Monitoring for Multiple Switch Topology
 12. Configuring Bonding for Maximum Throughput
 12.1	Maximum Throughput in a Single Switch Topology
 12.1.1		MT Bonding Mode Selection for Single Switch Topology
 12.1.2		MT Link Monitoring for Single Switch Topology
 12.2	Maximum Throughput in a Multiple Switch Topology
 12.2.1		MT Bonding Mode Selection for Multiple Switch Topology
 12.2.2		MT Link Monitoring for Multiple Switch Topology
 13. Switch Behavior Issues
 13.1	Link Establishment and Failover Delays
 13.2	Duplicated Incoming Packets
 14. Hardware Specific Considerations
 14.1	IBM BladeCenter
 15. Frequently Asked Questions
 16. Resources and Links
 1. Bonding Driver Installation
 ==============================
 	Most popular distro kernels ship with the bonding driver
 already available as a module and the ifenslave user level control
 program installed and ready for use. If your distro does not, or you
 have need to compile bonding from source (e.g., configuring and
 installing a mainline kernel from kernel.org), you'll need to perform
 the following steps:
 1.1 Configure and build the kernel with bonding
 -----------------------------------------------
 	The current version of the bonding driver is available in the
 drivers/net/bonding subdirectory of the most recent kernel source
 (which is available on http://kernel.org).  Most users "rolling their
 own" will want to use the most recent kernel from kernel.org.
 	Configure kernel with "make menuconfig" (or "make xconfig" or
 "make config"), then select "Bonding driver support" in the "Network
 device support" section.  It is recommended that you configure the
 driver as module since it is currently the only way to pass parameters
 to the driver or configure more than one bonding device.
 	Build and install the new kernel and modules, then continue
 below to install ifenslave.
 1.2 Install ifenslave Control Utility
 -------------------------------------
 	The ifenslave user level control program is included in the
 kernel source tree, in the file Documentation/networking/ifenslave.c.
 It is generally recommended that you use the ifenslave that
 corresponds to the kernel that you are using (either from the same
 source tree or supplied with the distro), however, ifenslave
 executables from older kernels should function (but features newer
 than the ifenslave release are not supported).  Running an ifenslave
 that is newer than the kernel is not supported, and may or may not
 work.
 	To install ifenslave, do the following:
 # gcc -Wall -O -I/usr/src/linux/include ifenslave.c -o ifenslave
 # cp ifenslave /sbin/ifenslave
 	If your kernel source is not in "/usr/src/linux," then replace
 "/usr/src/linux/include" in the above with the location of your kernel
 source include directory.
 	You may wish to back up any existing /sbin/ifenslave, or, for
 testing or informal use, tag the ifenslave to the kernel version
 (e.g., name the ifenslave executable /sbin/ifenslave-2.6.10).
 IMPORTANT NOTE:
 	If you omit the "-I" or specify an incorrect directory, you
 may end up with an ifenslave that is incompatible with the kernel
 you're trying to build it for.  Some distros (e.g., Red Hat from 7.1
 onwards) do not have /usr/include/linux symbolically linked to the
 default kernel source include directory.
 SECOND IMPORTANT NOTE:
 	If you plan to configure bonding using sysfs, you do not need
 to use ifenslave.
 2. Bonding Driver Options
 =========================
 	Options for the bonding driver are supplied as parameters to
 the bonding module at load time.  They may be given as command line
 arguments to the insmod or modprobe command, but are usually specified
 in either the /etc/modules.conf or /etc/modprobe.conf configuration
 file, or in a distro-specific configuration file (some of which are
 detailed in the next section).
 	The available bonding driver parameters are listed below. If a
 parameter is not specified the default value is used.  When initially
 configuring a bond, it is recommended "tail -f /var/log/messages" be
 run in a separate window to watch for bonding driver error messages.
 	It is critical that either the miimon or arp_interval and
 arp_ip_target parameters be specified, otherwise serious network
 degradation will occur during link failures.  Very few devices do not
 support at least miimon, so there is really no reason not to use it.
 	Options with textual values will accept either the text name
 or, for backwards compatibility, the option value.  E.g.,
 "mode=802.3ad" and "mode=4" set the same mode.
 	The parameters are as follows:
 arp_interval
 	Specifies the ARP link monitoring frequency in milliseconds.
 	The ARP monitor works by periodically checking the slave
 	devices to determine whether they have sent or received
 	traffic recently (the precise criteria depends upon the
 	bonding mode, and the state of the slave).  Regular traffic is
 	generated via ARP probes issued for the addresses specified by
 	the arp_ip_target option.
 	This behavior can be modified by the arp_validate option,
 	below.
 	If ARP monitoring is used in an etherchannel compatible mode
 	(modes 0 and 2), the switch should be configured in a mode
 	that evenly distributes packets across all links. If the
 	switch is configured to distribute the packets in an XOR
 	fashion, all replies from the ARP targets will be received on
 	the same link which could cause the other team members to
 	fail.  ARP monitoring should not be used in conjunction with
 	miimon.  A value of 0 disables ARP monitoring.  The default
 	value is 0.
 arp_ip_target
 	Specifies the IP addresses to use as ARP monitoring peers when
 	arp_interval is > 0.  These are the targets of the ARP request
 	sent to determine the health of the link to the targets.
 	Specify these values in ddd.ddd.ddd.ddd format.  Multiple IP
 	addresses must be separated by a comma.  At least one IP
 	address must be given for ARP monitoring to function.  The
 	maximum number of targets that can be specified is 16.  The
 	default value is no IP addresses.
 arp_validate
 	Specifies whether or not ARP probes and replies should be
 	validated in the active-backup mode.  This causes the ARP
 	monitor to examine the incoming ARP requests and replies, and
 	only consider a slave to be up if it is receiving the
 	appropriate ARP traffic.
 	Possible values are:
 	none or 0
 		No validation is performed.  This is the default.
 	active or 1
 		Validation is performed only for the active slave.
 	backup or 2
 		Validation is performed only for backup slaves.
 	all or 3
 		Validation is performed for all slaves.
 	For the active slave, the validation checks ARP replies to
 	confirm that they were generated by an arp_ip_target.  Since
 	backup slaves do not typically receive these replies, the
 	validation performed for backup slaves is on the ARP request
 	sent out via the active slave.  It is possible that some
 	switch or network configurations may result in situations
 	wherein the backup slaves do not receive the ARP requests; in
 	such a situation, validation of backup slaves must be
 	disabled.
 	This option is useful in network configurations in which
 	multiple bonding hosts are concurrently issuing ARPs to one or
 	more targets beyond a common switch.  Should the link between
 	the switch and target fail (but not the switch itself), the
 	probe traffic generated by the multiple bonding instances will
 	fool the standard ARP monitor into considering the links as
 	still up.  Use of the arp_validate option can resolve this, as
 	the ARP monitor will only consider ARP requests and replies
 	associated with its own instance of bonding.
 	This option was added in bonding version 3.1.0.
 downdelay
 	Specifies the time, in milliseconds, to wait before disabling
 	a slave after a link failure has been detected.  This option
 	is only valid for the miimon link monitor.  The downdelay
 	value should be a multiple of the miimon value; if not, it
 	will be rounded down to the nearest multiple.  The default
 	value is 0.
 lacp_rate
 	Option specifying the rate in which we'll ask our link partner
 	to transmit LACPDU packets in 802.3ad mode.  Possible values
 	are:
 	slow or 0
 		Request partner to transmit LACPDUs every 30 seconds
 	fast or 1
 		Request partner to transmit LACPDUs every 1 second
 	The default is slow.
 max_bonds
 	Specifies the number of bonding devices to create for this
 	instance of the bonding driver.  E.g., if max_bonds is 3, and
 	the bonding driver is not already loaded, then bond0, bond1
 	and bond2 will be created.  The default value is 1.
 miimon
 	Specifies the MII link monitoring frequency in milliseconds.
 	This determines how often the link state of each slave is
 	inspected for link failures.  A value of zero disables MII
 	link monitoring.  A value of 100 is a good starting point.
 	The use_carrier option, below, affects how the link state is
 	determined.  See the High Availability section for additional
 	information.  The default value is 0.
 mode
 	Specifies one of the bonding policies. The default is
 	balance-rr (round robin).  Possible values are:
 	balance-rr or 0
 		Round-robin policy: Transmit packets in sequential
 		order from the first available slave through the
 		last.  This mode provides load balancing and fault
 		tolerance.
 	active-backup or 1
 		Active-backup policy: Only one slave in the bond is
 		active.  A different slave becomes active if, and only
 		if, the active slave fails.  The bond's MAC address is
 		externally visible on only one port (network adapter)
 		to avoid confusing the switch.
 		In bonding version 2.6.2 or later, when a failover
 		occurs in active-backup mode, bonding will issue one
 		or more gratuitous ARPs on the newly active slave.
 		One gratuitous ARP is issued for the bonding master
 		interface and each VLAN interfaces configured above
 		it, provided that the interface has at least one IP
 		address configured.  Gratuitous ARPs issued for VLAN
 		interfaces are tagged with the appropriate VLAN id.
 		This mode provides fault tolerance.  The primary
 		option, documented below, affects the behavior of this
 		mode.
 	balance-xor or 2
 		XOR policy: Transmit based on the selected transmit
 		hash policy.  The default policy is a simple [(source
 		MAC address XOR'd with destination MAC address) modulo
 		slave count].  Alternate transmit policies may be
 		selected via the xmit_hash_policy option, described
 		below.
 		This mode provides load balancing and fault tolerance.
 	broadcast or 3
 		Broadcast policy: transmits everything on all slave
 		interfaces.  This mode provides fault tolerance.
 	802.3ad or 4
 		IEEE 802.3ad Dynamic link aggregation.  Creates
 		aggregation groups that share the same speed and
 		duplex settings.  Utilizes all slaves in the active
 		aggregator according to the 802.3ad specification.
 		Slave selection for outgoing traffic is done according
 		to the transmit hash policy, which may be changed from
 		the default simple XOR policy via the xmit_hash_policy
 		option, documented below.  Note that not all transmit
 		policies may be 802.3ad compliant, particularly in
 		regards to the packet mis-ordering requirements of
 		section 43.2.4 of the 802.3ad standard.  Differing
 		peer implementations will have varying tolerances for
 		noncompliance.
 		Prerequisites:
 		1. Ethtool support in the base drivers for retrieving
 		the speed and duplex of each slave.
 		2. A switch that supports IEEE 802.3ad Dynamic link
 		aggregation.
 		Most switches will require some type of configuration
 		to enable 802.3ad mode.
 	balance-tlb or 5
 		Adaptive transmit load balancing: channel bonding that
 		does not require any special switch support.  The
 		outgoing traffic is distributed according to the
 		current load (computed relative to the speed) on each
 		slave.  Incoming traffic is received by the current
 		slave.  If the receiving slave fails, another slave
 		takes over the MAC address of the failed receiving
 		slave.
 		Prerequisite:
 		Ethtool support in the base drivers for retrieving the
 		speed of each slave.
 	balance-alb or 6
 		Adaptive load balancing: includes balance-tlb plus
 		receive load balancing (rlb) for IPV4 traffic, and
 		does not require any special switch support.  The
 		receive load balancing is achieved by ARP negotiation.
 		The bonding driver intercepts the ARP Replies sent by
 		the local system on their way out and overwrites the
 		source hardware address with the unique hardware
 		address of one of the slaves in the bond such that
 		different peers use different hardware addresses for
 		the server.
 		Receive traffic from connections created by the server
 		is also balanced.  When the local system sends an ARP
 		Request the bonding driver copies and saves the peer's
 		IP information from the ARP packet.  When the ARP
 		Reply arrives from the peer, its hardware address is
 		retrieved and the bonding driver initiates an ARP
 		reply to this peer assigning it to one of the slaves
 		in the bond.  A problematic outcome of using ARP
 		negotiation for balancing is that each time that an
 		ARP request is broadcast it uses the hardware address
 		of the bond.  Hence, peers learn the hardware address
 		of the bond and the balancing of receive traffic
 		collapses to the current slave.  This is handled by
 		sending updates (ARP Replies) to all the peers with
 		their individually assigned hardware address such that
 		the traffic is redistributed.  Receive traffic is also
 		redistributed when a new slave is added to the bond
 		and when an inactive slave is re-activated.  The
 		receive load is distributed sequentially (round robin)
 		among the group of highest speed slaves in the bond.
 		When a link is reconnected or a new slave joins the
 		bond the receive traffic is redistributed among all
 		active slaves in the bond by initiating ARP Replies
 		with the selected MAC address to each of the
 		clients. The updelay parameter (detailed below) must
 		be set to a value equal or greater than the switch's
 		forwarding delay so that the ARP Replies sent to the
 		peers will not be blocked by the switch.
 		Prerequisites:
 		1. Ethtool support in the base drivers for retrieving
 		the speed of each slave.
 		2. Base driver support for setting the hardware
 		address of a device while it is open.  This is
 		required so that there will always be one slave in the
 		team using the bond hardware address (the
 		curr_active_slave) while having a unique hardware
 		address for each slave in the bond.  If the
 		curr_active_slave fails its hardware address is
 		swapped with the new curr_active_slave that was
 		chosen.
 primary
 	A string (eth0, eth2, etc) specifying which slave is the
 	primary device.  The specified device will always be the
 	active slave while it is available.  Only when the primary is
 	off-line will alternate devices be used.  This is useful when
 	one slave is preferred over another, e.g., when one slave has
 	higher throughput than another.
 	The primary option is only valid for active-backup mode.
 updelay
 	Specifies the time, in milliseconds, to wait before enabling a
 	slave after a link recovery has been detected.  This option is
 	only valid for the miimon link monitor.  The updelay value
 	should be a multiple of the miimon value; if not, it will be
 	rounded down to the nearest multiple.  The default value is 0.
 use_carrier
 	Specifies whether or not miimon should use MII or ETHTOOL
 	ioctls vs. netif_carrier_ok() to determine the link
 	status. The MII or ETHTOOL ioctls are less efficient and
 	utilize a deprecated calling sequence within the kernel.  The
 	netif_carrier_ok() relies on the device driver to maintain its
 	state with netif_carrier_on/off; at this writing, most, but
 	not all, device drivers support this facility.
 	If bonding insists that the link is up when it should not be,
 	it may be that your network device driver does not support
 	netif_carrier_on/off.  The default state for netif_carrier is
 	"carrier on," so if a driver does not support netif_carrier,
 	it will appear as if the link is always up.  In this case,
 	setting use_carrier to 0 will cause bonding to revert to the
 	MII / ETHTOOL ioctl method to determine the link state.
 	A value of 1 enables the use of netif_carrier_ok(), a value of
 	0 will use the deprecated MII / ETHTOOL ioctls.  The default
 	value is 1.
 xmit_hash_policy
 	Selects the transmit hash policy to use for slave selection in
 	balance-xor and 802.3ad modes.  Possible values are:
 	layer2
 		Uses XOR of hardware MAC addresses to generate the
 		hash.  The formula is
 		(source MAC XOR destination MAC) modulo slave count
 		This algorithm will place all traffic to a particular
 		network peer on the same slave.
 		This algorithm is 802.3ad compliant.
 	layer3+4
 		This policy uses upper layer protocol information,
 		when available, to generate the hash.  This allows for
 		traffic to a particular network peer to span multiple
 		slaves, although a single connection will not span
 		multiple slaves.
 		The formula for unfragmented TCP and UDP packets is
 		((source port XOR dest port) XOR
 			 ((source IP XOR dest IP) AND 0xffff)
 				modulo slave count
 		For fragmented TCP or UDP packets and all other IP
 		protocol traffic, the source and destination port
 		information is omitted.  For non-IP traffic, the
 		formula is the same as for the layer2 transmit hash
 		policy.
 		This policy is intended to mimic the behavior of
 		certain switches, notably Cisco switches with PFC2 as
 		well as some Foundry and IBM products.
 		This algorithm is not fully 802.3ad compliant.  A
 		single TCP or UDP conversation containing both
 		fragmented and unfragmented packets will see packets
 		striped across two interfaces.  This may result in out
 		of order delivery.  Most traffic types will not meet
 		this criteria, as TCP rarely fragments traffic, and
 		most UDP traffic is not involved in extended
 		conversations.  Other implementations of 802.3ad may
 		or may not tolerate this noncompliance.
 	The default value is layer2.  This option was added in bonding
 version 2.6.3.  In earlier versions of bonding, this parameter does
 not exist, and the layer2 policy is the only policy.
 3. Configuring Bonding Devices
 ==============================
 	You can configure bonding using either your distro's network
 initialization scripts, or manually using either ifenslave or the
 sysfs interface.  Distros generally use one of two packages for the
 network initialization scripts: initscripts or sysconfig.  Recent
 versions of these packages have support for bonding, while older
 versions do not.
 	We will first describe the options for configuring bonding for
 distros using versions of initscripts and sysconfig with full or
 partial support for bonding, then provide information on enabling
 bonding without support from the network initialization scripts (i.e.,
 older versions of initscripts or sysconfig).
 	If you're unsure whether your distro uses sysconfig or
 initscripts, or don't know if it's new enough, have no fear.
 Determining this is fairly straightforward.
 	First, issue the command:
 $ rpm -qf /sbin/ifup
 	It will respond with a line of text starting with either
 "initscripts" or "sysconfig," followed by some numbers.  This is the
 package that provides your network initialization scripts.
 	Next, to determine if your installation supports bonding,
 issue the command:
 $ grep ifenslave /sbin/ifup
 	If this returns any matches, then your initscripts or
 sysconfig has support for bonding.
 3.1 Configuration with Sysconfig Support
 ----------------------------------------
 	This section applies to distros using a version of sysconfig
 with bonding support, for example, SuSE Linux Enterprise Server 9.
 	SuSE SLES 9's networking configuration system does support
 bonding, however, at this writing, the YaST system configuration
 front end does not provide any means to work with bonding devices.
 Bonding devices can be managed by hand, however, as follows.
 	First, if they have not already been configured, configure the
 slave devices.  On SLES 9, this is most easily done by running the
 yast2 sysconfig configuration utility.  The goal is for to create an
 ifcfg-id file for each slave device.  The simplest way to accomplish
 this is to configure the devices for DHCP (this is only to get the
 file ifcfg-id file created; see below for some issues with DHCP).  The
 name of the configuration file for each device will be of the form:
 ifcfg-id-xx:xx:xx:xx:xx:xx
 	Where the "xx" portion will be replaced with the digits from
 the device's permanent MAC address.
 	Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been
 created, it is necessary to edit the configuration files for the slave
 devices (the MAC addresses correspond to those of the slave devices).
 Before editing, the file will contain multiple lines, and will look
 something like this:
 BOOTPROTO='dhcp'
 STARTMODE='on'
 USERCTL='no'
 UNIQUE='XNzu.WeZGOGF+4wE'
 _nm_name='bus-pci-0001:61:01.0'
 	Change the BOOTPROTO and STARTMODE lines to the following:
 BOOTPROTO='none'
 STARTMODE='off'
 	Do not alter the UNIQUE or _nm_name lines.  Remove any other
 lines (USERCTL, etc).
 	Once the ifcfg-id-xx:xx:xx:xx:xx:xx files have been modified,
 it's time to create the configuration file for the bonding device
 itself.  This file is named ifcfg-bondX, where X is the number of the
 bonding device to create, starting at 0.  The first such file is
 ifcfg-bond0, the second is ifcfg-bond1, and so on.  The sysconfig
 network configuration system will correctly start multiple instances
 of bonding.
 	The contents of the ifcfg-bondX file is as follows:
 BOOTPROTO="static"
 BROADCAST="10.0.2.255"
 IPADDR="10.0.2.10"
 NETMASK="255.255.0.0"
 NETWORK="10.0.2.0"
 REMOTE_IPADDR=""
 STARTMODE="onboot"
 BONDING_MASTER="yes"
 BONDING_MODULE_OPTS="mode=active-backup miimon=100"
 BONDING_SLAVE0="eth0"
 BONDING_SLAVE1="bus-pci-0000:06:08.1"
 	Replace the sample BROADCAST, IPADDR, NETMASK and NETWORK
 values with the appropriate values for your network.
 	The STARTMODE specifies when the device is brought online.
 The possible values are:
 	onboot:	 The device is started at boot time.  If you're not
 		 sure, this is probably what you want.
 	manual:	 The device is started only when ifup is called
 		 manually.  Bonding devices may be configured this
 		 way if you do not wish them to start automatically
 		 at boot for some reason.
 	hotplug: The device is started by a hotplug event.  This is not
 		 a valid choice for a bonding device.
 	off or ignore: The device configuration is ignored.
 	The line BONDING_MASTER='yes' indicates that the device is a
 bonding master device.  The only useful value is "yes."
 	The contents of BONDING_MODULE_OPTS are supplied to the
 instance of the bonding module for this device.  Specify the options
 for the bonding mode, link monitoring, and so on here.  Do not include
 the max_bonds bonding parameter; this will confuse the configuration
 system if you have multiple bonding devices.
 	Finally, supply one BONDING_SLAVEn="slave device" for each
 slave.  where "n" is an increasing value, one for each slave.  The
 "slave device" is either an interface name, e.g., "eth0", or a device
 specifier for the network device.  The interface name is easier to
 find, but the ethN names are subject to change at boot time if, e.g.,
 a device early in the sequence has failed.  The device specifiers
 (bus-pci-0000:06:08.1 in the example above) specify the physical
 network device, and will not change unless the device's bus location
 changes (for example, it is moved from one PCI slot to another).  The
 example above uses one of each type for demonstration purposes; most
 configurations will choose one or the other for all slave devices.
 	When all configuration files have been modified or created,
 networking must be restarted for the configuration changes to take
 effect.  This can be accomplished via the following:
 # /etc/init.d/network restart
 	Note that the network control script (/sbin/ifdown) will
 remove the bonding module as part of the network shutdown processing,
 so it is not necessary to remove the module by hand if, e.g., the
 module parameters have changed.
 	Also, at this writing, YaST/YaST2 will not manage bonding
 devices (they do not show bonding interfaces on its list of network
 devices).  It is necessary to edit the configuration file by hand to
 change the bonding configuration.
 	Additional general options and details of the ifcfg file
 format can be found in an example ifcfg template file:
 /etc/sysconfig/network/ifcfg.template
 	Note that the template does not document the various BONDING_
 settings described above, but does describe many of the other options.
 3.1.1 Using DHCP with Sysconfig
 -------------------------------
 	Under sysconfig, configuring a device with BOOTPROTO='dhcp'
 will cause it to query DHCP for its IP address information.  At this
 writing, this does not function for bonding devices; the scripts
 attempt to obtain the device address from DHCP prior to adding any of
 the slave devices.  Without active slaves, the DHCP requests are not
 sent to the network.
 3.1.2 Configuring Multiple Bonds with Sysconfig
 -----------------------------------------------
 	The sysconfig network initialization system is capable of
 handling multiple bonding devices.  All that is necessary is for each
 bonding instance to have an appropriately configured ifcfg-bondX file
 (as described above).  Do not specify the "max_bonds" parameter to any
 instance of bonding, as this will confuse sysconfig.  If you require
 multiple bonding devices with identical parameters, create multiple
 ifcfg-bondX files.
 	Because the sysconfig scripts supply the bonding module
 options in the ifcfg-bondX file, it is not necessary to add them to
 the system /etc/modules.conf or /etc/modprobe.conf configuration file.
 3.2 Configuration with Initscripts Support
 ------------------------------------------
 	This section applies to distros using a version of initscripts
 with bonding support, for example, Red Hat Linux 9 or Red Hat
 Enterprise Linux version 3 or 4.  On these systems, the network
 initialization scripts have some knowledge of bonding, and can be
 configured to control bonding devices.
 	These distros will not automatically load the network adapter
 driver unless the ethX device is configured with an IP address.
 Because of this constraint, users must manually configure a
 network-script file for all physical adapters that will be members of
 a bondX link.  Network script files are located in the directory:
 /etc/sysconfig/network-scripts
 	The file name must be prefixed with "ifcfg-eth" and suffixed
 with the adapter's physical adapter number.  For example, the script
 for eth0 would be named /etc/sysconfig/network-scripts/ifcfg-eth0.
 Place the following text in the file:
 DEVICE=eth0
 USERCTL=no
 ONBOOT=yes
 MASTER=bond0
 SLAVE=yes
 BOOTPROTO=none
 	The DEVICE= line will be different for every ethX device and
 must correspond with the name of the file, i.e., ifcfg-eth1 must have
 a device line of DEVICE=eth1.  The setting of the MASTER= line will
 also depend on the final bonding interface name chosen for your bond.
 As with other network devices, these typically start at 0, and go up
 one for each device, i.e., the first bonding instance is bond0, the
 second is bond1, and so on.
 	Next, create a bond network script.  The file name for this
 script will be /etc/sysconfig/network-scripts/ifcfg-bondX where X is
 the number of the bond.  For bond0 the file is named "ifcfg-bond0",
 for bond1 it is named "ifcfg-bond1", and so on.  Within that file,
 place the following text:
 DEVICE=bond0
 IPADDR=192.168.1.1
 NETMASK=255.255.255.0
 NETWORK=192.168.1.0
 BROADCAST=192.168.1.255
 ONBOOT=yes
 BOOTPROTO=none
 USERCTL=no
 	Be sure to change the networking specific lines (IPADDR,
 NETMASK, NETWORK and BROADCAST) to match your network configuration.
 	Finally, it is necessary to edit /etc/modules.conf (or
 /etc/modprobe.conf, depending upon your distro) to load the bonding
 module with your desired options when the bond0 interface is brought
 up.  The following lines in /etc/modules.conf (or modprobe.conf) will
 load the bonding module, and select its options:
 alias bond0 bonding
 options bond0 mode=balance-alb miimon=100
 	Replace the sample parameters with the appropriate set of
 options for your configuration.
 	Finally run "/etc/rc.d/init.d/network restart" as root.  This
 will restart the networking subsystem and your bond link should be now
 up and running.
 3.2.1 Using DHCP with Initscripts
 ---------------------------------
 	Recent versions of initscripts (the version supplied with
 Fedora Core 3 and Red Hat Enterprise Linux 4 is reported to work) do
 have support for assigning IP information to bonding devices via DHCP.
 	To configure bonding for DHCP, configure it as described
 above, except replace the line "BOOTPROTO=none" with "BOOTPROTO=dhcp"
 and add a line consisting of "TYPE=Bonding".  Note that the TYPE value
 is case sensitive.
 3.2.2 Configuring Multiple Bonds with Initscripts
 -------------------------------------------------
 	At this writing, the initscripts package does not directly
 support loading the bonding driver multiple times, so the process for
 doing so is the same as described in the "Configuring Multiple Bonds
 Manually" section, below.
 	NOTE: It has been observed that some Red Hat supplied kernels
 are apparently unable to rename modules at load time (the "-o bond1"
 part).  Attempts to pass that option to modprobe will produce an
 "Operation not permitted" error.  This has been reported on some
 Fedora Core kernels, and has been seen on RHEL 4 as well.  On kernels
 exhibiting this problem, it will be impossible to configure multiple
 bonds with differing parameters.
 3.3 Configuring Bonding Manually with Ifenslave
 -----------------------------------------------
 	This section applies to distros whose network initialization
 scripts (the sysconfig or initscripts package) do not have specific
 knowledge of bonding.  One such distro is SuSE Linux Enterprise Server
 version 8.
 	The general method for these systems is to place the bonding
 module parameters into /etc/modules.conf or /etc/modprobe.conf (as
 appropriate for the installed distro), then add modprobe and/or
 ifenslave commands to the system's global init script.  The name of
 the global init script differs; for sysconfig, it is
 /etc/init.d/boot.local and for initscripts it is /etc/rc.d/rc.local.
 	For example, if you wanted to make a simple bond of two e100
 devices (presumed to be eth0 and eth1), and have it persist across
 reboots, edit the appropriate file (/etc/init.d/boot.local or
 /etc/rc.d/rc.local), and add the following:
 modprobe bonding mode=balance-alb miimon=100
 modprobe e100
 ifconfig bond0 192.168.1.1 netmask 255.255.255.0 up
 ifenslave bond0 eth0
 ifenslave bond0 eth1
 	Replace the example bonding module parameters and bond0
 network configuration (IP address, netmask, etc) with the appropriate
 values for your configuration.
 	Unfortunately, this method will not provide support for the
 ifup and ifdown scripts on the bond devices.  To reload the bonding
 configuration, it is necessary to run the initialization script, e.g.,
 # /etc/init.d/boot.local
 	or
 # /etc/rc.d/rc.local
 	It may be desirable in such a case to create a separate script
 which only initializes the bonding configuration, then call that
 separate script from within boot.local.  This allows for bonding to be
 enabled without re-running the entire global init script.
 	To shut down the bonding devices, it is necessary to first
 mark the bonding device itself as being down, then remove the
 appropriate device driver modules.  For our example above, you can do
 the following:
 # ifconfig bond0 down
 # rmmod bonding
 # rmmod e100
 	Again, for convenience, it may be desirable to create a script
 with these commands.
 3.3.1 Configuring Multiple Bonds Manually
 -----------------------------------------
 	This section contains information on configuring multiple
 bonding devices with differing options for those systems whose network
 initialization scripts lack support for configuring multiple bonds.
 	If you require multiple bonding devices, but all with the same
 options, you may wish to use the "max_bonds" module parameter,
 documented above.
 	To create multiple bonding devices with differing options, it
 is necessary to load the bonding driver multiple times.  Note that
 current versions of the sysconfig network initialization scripts
 handle this automatically; if your distro uses these scripts, no
 special action is needed.  See the section Configuring Bonding
 Devices, above, if you're not sure about your network initialization
 scripts.
 	To load multiple instances of the module, it is necessary to
 specify a different name for each instance (the module loading system
 requires that every loaded module, even multiple instances of the same
 module, have a unique name).  This is accomplished by supplying
 multiple sets of bonding options in /etc/modprobe.conf, for example:
 alias bond0 bonding
 options bond0 -o bond0 mode=balance-rr miimon=100
 alias bond1 bonding
 options bond1 -o bond1 mode=balance-alb miimon=50
 	will load the bonding module two times.  The first instance is
 named "bond0" and creates the bond0 device in balance-rr mode with an
 miimon of 100.  The second instance is named "bond1" and creates the
 bond1 device in balance-alb mode with an miimon of 50.
 	In some circumstances (typically with older distributions),
 the above does not work, and the second bonding instance never sees
 its options.  In that case, the second options line can be substituted
 as follows:
 install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
 	mode=balance-alb miimon=50
 	This may be repeated any number of times, specifying a new and
 unique name in place of bond1 for each subsequent instance.
 3.4 Configuring Bonding Manually via Sysfs
 ------------------------------------------
 	Starting with version 3.0, Channel Bonding may be configured
 via the sysfs interface.  This interface allows dynamic configuration
 of all bonds in the system without unloading the module.  It also
 allows for adding and removing bonds at runtime.  Ifenslave is no
 longer required, though it is still supported.
 	Use of the sysfs interface allows you to use multiple bonds
 with different configurations without having to reload the module.
 It also allows you to use multiple, differently configured bonds when
 bonding is compiled into the kernel.
 	You must have the sysfs filesystem mounted to configure
 bonding this way.  The examples in this document assume that you
 are using the standard mount point for sysfs, e.g. /sys.  If your
 sysfs filesystem is mounted elsewhere, you will need to adjust the
 example paths accordingly.
 Creating and Destroying Bonds
 -----------------------------
 To add a new bond foo:
 # echo +foo > /sys/class/net/bonding_masters
 To remove an existing bond bar:
 # echo -bar > /sys/class/net/bonding_masters
 To show all existing bonds:
 # cat /sys/class/net/bonding_masters
 NOTE: due to 4K size limitation of sysfs files, this list may be
 truncated if you have more than a few hundred bonds.  This is unlikely
 to occur under normal operating conditions.
 Adding and Removing Slaves
 --------------------------
 	Interfaces may be enslaved to a bond using the file
 /sys/class/net/<bond>/bonding/slaves.  The semantics for this file
 are the same as for the bonding_masters file.
 To enslave interface eth0 to bond bond0:
 # ifconfig bond0 up
 # echo +eth0 > /sys/class/net/bond0/bonding/slaves
 To free slave eth0 from bond bond0:
 # echo -eth0 > /sys/class/net/bond0/bonding/slaves
 	NOTE: The bond must be up before slaves can be added.  All
 slaves are freed when the interface is brought down.
 	When an interface is enslaved to a bond, symlinks between the
 two are created in the sysfs filesystem.  In this case, you would get
 /sys/class/net/bond0/slave_eth0 pointing to /sys/class/net/eth0, and
 /sys/class/net/eth0/master pointing to /sys/class/net/bond0.
 	This means that you can tell quickly whether or not an
 interface is enslaved by looking for the master symlink.  Thus:
 # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves
 will free eth0 from whatever bond it is enslaved to, regardless of
 the name of the bond interface.
 Changing a Bond's Configuration
 -------------------------------
 	Each bond may be configured individually by manipulating the
 files located in /sys/class/net/<bond name>/bonding
 	The names of these files correspond directly with the command-
-line parameters described elsewhere in in this file, and, with the
+line parameters described elsewhere in this file, and, with the
 exception of arp_ip_target, they accept the same values.  To see the
 current setting, simply cat the appropriate file.
 	A few examples will be given here; for specific usage
 guidelines for each parameter, see the appropriate section in this
 document.
 To configure bond0 for balance-alb mode:
 # ifconfig bond0 down
 # echo 6 > /sys/class/net/bond0/bonding/mode
 - or -
 # echo balance-alb > /sys/class/net/bond0/bonding/mode
 	NOTE: The bond interface must be down before the mode can be
 changed.
 To enable MII monitoring on bond0 with a 1 second interval:
 # echo 1000 > /sys/class/net/bond0/bonding/miimon
 	NOTE: If ARP monitoring is enabled, it will disabled when MII
 monitoring is enabled, and vice-versa.
 To add ARP targets:
 # echo +192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
 # echo +192.168.0.101 > /sys/class/net/bond0/bonding/arp_ip_target
 	NOTE:  up to 10 target addresses may be specified.
 To remove an ARP target:
 # echo -192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
 Example Configuration
 ---------------------
 	We begin with the same example that is shown in section 3.3,
 executed with sysfs, and without using ifenslave.
 	To make a simple bond of two e100 devices (presumed to be eth0
 and eth1), and have it persist across reboots, edit the appropriate
 file (/etc/init.d/boot.local or /etc/rc.d/rc.local), and add the
 following:
 modprobe bonding
 modprobe e100
 echo balance-alb > /sys/class/net/bond0/bonding/mode
 ifconfig bond0 192.168.1.1 netmask 255.255.255.0 up
 echo 100 > /sys/class/net/bond0/bonding/miimon
 echo +eth0 > /sys/class/net/bond0/bonding/slaves
 echo +eth1 > /sys/class/net/bond0/bonding/slaves
 	To add a second bond, with two e1000 interfaces in
 active-backup mode, using ARP monitoring, add the following lines to
 your init script:
 modprobe e1000
 echo +bond1 > /sys/class/net/bonding_masters
 echo active-backup > /sys/class/net/bond1/bonding/mode
 ifconfig bond1 192.168.2.1 netmask 255.255.255.0 up
 echo +192.168.2.100 /sys/class/net/bond1/bonding/arp_ip_target
 echo 2000 > /sys/class/net/bond1/bonding/arp_interval
 echo +eth2 > /sys/class/net/bond1/bonding/slaves
 echo +eth3 > /sys/class/net/bond1/bonding/slaves
 4. Querying Bonding Configuration 
 =================================
 4.1 Bonding Configuration
 -------------------------
 	Each bonding device has a read-only file residing in the
 /proc/net/bonding directory.  The file contents include information
 about the bonding configuration, options and state of each slave.
 	For example, the contents of /proc/net/bonding/bond0 after the
 driver is loaded with parameters of mode=0 and miimon=1000 is
 generally as follows:
 	Ethernet Channel Bonding Driver: 2.6.1 (October 29, 2004)
        Bonding Mode: load balancing (round-robin)
        Currently Active Slave: eth0
        MII Status: up
        MII Polling Interval (ms): 1000
        Up Delay (ms): 0
        Down Delay (ms): 0
        Slave Interface: eth1
        MII Status: up
        Link Failure Count: 1
        Slave Interface: eth0
        MII Status: up
        Link Failure Count: 1
 	The precise format and contents will change depending upon the
 bonding configuration, state, and version of the bonding driver.
 4.2 Network configuration
 -------------------------
 	The network configuration can be inspected using the ifconfig
 command.  Bonding devices will have the MASTER flag set; Bonding slave
 devices will have the SLAVE flag set.  The ifconfig output does not
 contain information on which slaves are associated with which masters.
 	In the example below, the bond0 interface is the master
 (MASTER) while eth0 and eth1 are slaves (SLAVE). Notice all slaves of
 bond0 have the same MAC address (HWaddr) as bond0 for all modes except
 TLB and ALB that require a unique MAC address for each slave.
 # /sbin/ifconfig
 bond0     Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:7224794 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3286647 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:0
 eth0      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3573025 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643167 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:100
          Interrupt:10 Base address:0x1080
 eth1      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3651769 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643480 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          Interrupt:9 Base address:0x1400
 5. Switch Configuration
 =======================
 	For this section, "switch" refers to whatever system the
 bonded devices are directly connected to (i.e., where the other end of
 the cable plugs into).  This may be an actual dedicated switch device,
 or it may be another regular system (e.g., another computer running
 Linux),
 	The active-backup, balance-tlb and balance-alb modes do not
 require any specific configuration of the switch.
 	The 802.3ad mode requires that the switch have the appropriate
 ports configured as an 802.3ad aggregation.  The precise method used
 to configure this varies from switch to switch, but, for example, a
 Cisco 3550 series switch requires that the appropriate ports first be
 grouped together in a single etherchannel instance, then that
 etherchannel is set to mode "lacp" to enable 802.3ad (instead of
 standard EtherChannel).
 	The balance-rr, balance-xor and broadcast modes generally
 require that the switch have the appropriate ports grouped together.
 The nomenclature for such a group differs between switches, it may be
 called an "etherchannel" (as in the Cisco example, above), a "trunk
 group" or some other similar variation.  For these modes, each switch
 will also have its own configuration options for the switch's transmit
 policy to the bond.  Typical choices include XOR of either the MAC or
 IP addresses.  The transmit policy of the two peers does not need to
 match.  For these three modes, the bonding mode really selects a
 transmit policy for an EtherChannel group; all three will interoperate
 with another EtherChannel group.
 6. 802.1q VLAN Support
 ======================
 	It is possible to configure VLAN devices over a bond interface
 using the 8021q driver.  However, only packets coming from the 8021q
 driver and passing through bonding will be tagged by default.  Self
 generated packets, for example, bonding's learning packets or ARP
 packets generated by either ALB mode or the ARP monitor mechanism, are
 tagged internally by bonding itself.  As a result, bonding must
 "learn" the VLAN IDs configured above it, and use those IDs to tag
 self generated packets.
 	For reasons of simplicity, and to support the use of adapters
 that can do VLAN hardware acceleration offloading, the bonding
 interface declares itself as fully hardware offloading capable, it gets
 the add_vid/kill_vid notifications to gather the necessary
 information, and it propagates those actions to the slaves.  In case
 of mixed adapter types, hardware accelerated tagged packets that
 should go through an adapter that is not offloading capable are
 "un-accelerated" by the bonding driver so the VLAN tag sits in the
 regular location.
 	VLAN interfaces *must* be added on top of a bonding interface
 only after enslaving at least one slave.  The bonding interface has a
 hardware address of 00:00:00:00:00:00 until the first slave is added.
 If the VLAN interface is created prior to the first enslavement, it
 would pick up the all-zeroes hardware address.  Once the first slave
 is attached to the bond, the bond device itself will pick up the
 slave's hardware address, which is then available for the VLAN device.
 	Also, be aware that a similar problem can occur if all slaves
 are released from a bond that still has one or more VLAN interfaces on
 top of it.  When a new slave is added, the bonding interface will
 obtain its hardware address from the first slave, which might not
 match the hardware address of the VLAN interfaces (which was
 ultimately copied from an earlier slave).
 	There are two methods to insure that the VLAN device operates
 with the correct hardware address if all slaves are removed from a
 bond interface:
 	1. Remove all VLAN interfaces then recreate them
 	2. Set the bonding interface's hardware address so that it
 matches the hardware address of the VLAN interfaces.
 	Note that changing a VLAN interface's HW address would set the
 underlying device -- i.e. the bonding interface -- to promiscuous
 mode, which might not be what you want.
 7. Link Monitoring
 ==================
 	The bonding driver at present supports two schemes for
 monitoring a slave device's link state: the ARP monitor and the MII
 monitor.
 	At the present time, due to implementation restrictions in the
 bonding driver itself, it is not possible to enable both ARP and MII
 monitoring simultaneously.
 7.1 ARP Monitor Operation
 -------------------------
 	The ARP monitor operates as its name suggests: it sends ARP
 queries to one or more designated peer systems on the network, and
 uses the response as an indication that the link is operating.  This
 gives some assurance that traffic is actually flowing to and from one
 or more peers on the local network.
 	The ARP monitor relies on the device driver itself to verify
 that traffic is flowing.  In particular, the driver must keep up to
 date the last receive time, dev->last_rx, and transmit start time,
 dev->trans_start.  If these are not updated by the driver, then the
 ARP monitor will immediately fail any slaves using that driver, and
 those slaves will stay down.  If networking monitoring (tcpdump, etc)
 shows the ARP requests and replies on the network, then it may be that
 your device driver is not updating last_rx and trans_start.
 7.2 Configuring Multiple ARP Targets
 ------------------------------------
 	While ARP monitoring can be done with just one target, it can
 be useful in a High Availability setup to have several targets to
 monitor.  In the case of just one target, the target itself may go
 down or have a problem making it unresponsive to ARP requests.  Having
 an additional target (or several) increases the reliability of the ARP
 monitoring.
 	Multiple ARP targets must be separated by commas as follows:
 # example options for ARP monitoring with three targets
 alias bond0 bonding
 options bond0 arp_interval=60 arp_ip_target=192.168.0.1,192.168.0.3,192.168.0.9
 	For just a single target the options would resemble:
 # example options for ARP monitoring with one target
 alias bond0 bonding
 options bond0 arp_interval=60 arp_ip_target=192.168.0.100
 7.3 MII Monitor Operation
 -------------------------
 	The MII monitor monitors only the carrier state of the local
 network interface.  It accomplishes this in one of three ways: by
 depending upon the device driver to maintain its carrier state, by
 querying the device's MII registers, or by making an ethtool query to
 the device.
 	If the use_carrier module parameter is 1 (the default value),
 then the MII monitor will rely on the driver for carrier state
 information (via the netif_carrier subsystem).  As explained in the
 use_carrier parameter information, above, if the MII monitor fails to
 detect carrier loss on the device (e.g., when the cable is physically
 disconnected), it may be that the driver does not support
 netif_carrier.
 	If use_carrier is 0, then the MII monitor will first query the
 device's (via ioctl) MII registers and check the link state.  If that
 request fails (not just that it returns carrier down), then the MII
 monitor will make an ethtool ETHOOL_GLINK request to attempt to obtain
 the same information.  If both methods fail (i.e., the driver either
 does not support or had some error in processing both the MII register
 and ethtool requests), then the MII monitor will assume the link is
 up.
 8. Potential Sources of Trouble
 ===============================
 8.1 Adventures in Routing
 -------------------------
 	When bonding is configured, it is important that the slave
 devices not have routes that supersede routes of the master (or,
 generally, not have routes at all).  For example, suppose the bonding
 device bond0 has two slaves, eth0 and eth1, and the routing table is
 as follows:
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
 10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 eth0
 10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 eth1
 10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 bond0
 127.0.0.0       0.0.0.0         255.0.0.0       U        40 0          0 lo
 	This routing configuration will likely still update the
 receive/transmit times in the driver (needed by the ARP monitor), but
 may bypass the bonding driver (because outgoing traffic to, in this
 case, another host on network 10 would use eth0 or eth1 before bond0).
 	The ARP monitor (and ARP itself) may become confused by this
 configuration, because ARP requests (generated by the ARP monitor)
 will be sent on one interface (bond0), but the corresponding reply
 will arrive on a different interface (eth0).  This reply looks to ARP
 as an unsolicited ARP reply (because ARP matches replies on an
 interface basis), and is discarded.  The MII monitor is not affected
 by the state of the routing table.
 	The solution here is simply to insure that slaves do not have
 routes of their own, and if for some reason they must, those routes do
 not supersede routes of their master.  This should generally be the
 case, but unusual configurations or errant manual or automatic static
 route additions may cause trouble.
 8.2 Ethernet Device Renaming
 ----------------------------
 	On systems with network configuration scripts that do not
 associate physical devices directly with network interface names (so
 that the same physical device always has the same "ethX" name), it may
 be necessary to add some special logic to either /etc/modules.conf or
 /etc/modprobe.conf (depending upon which is installed on the system).
 	For example, given a modules.conf containing the following:
 alias bond0 bonding
 options bond0 mode=some-mode miimon=50
 alias eth0 tg3
 alias eth1 tg3
 alias eth2 e1000
 alias eth3 e1000
 	If neither eth0 and eth1 are slaves to bond0, then when the
 bond0 interface comes up, the devices may end up reordered.  This
 happens because bonding is loaded first, then its slave device's
 drivers are loaded next.  Since no other drivers have been loaded,
 when the e1000 driver loads, it will receive eth0 and eth1 for its
 devices, but the bonding configuration tries to enslave eth2 and eth3
 (which may later be assigned to the tg3 devices).
 	Adding the following:
 add above bonding e1000 tg3
 	causes modprobe to load e1000 then tg3, in that order, when
 bonding is loaded.  This command is fully documented in the
 modules.conf manual page.
 	On systems utilizing modprobe.conf (or modprobe.conf.local),
 an equivalent problem can occur.  In this case, the following can be
 added to modprobe.conf (or modprobe.conf.local, as appropriate), as
 follows (all on one line; it has been split here for clarity):
 install bonding /sbin/modprobe tg3; /sbin/modprobe e1000;
 	/sbin/modprobe --ignore-install bonding
 	This will, when loading the bonding module, rather than
 performing the normal action, instead execute the provided command.
 This command loads the device drivers in the order needed, then calls
 modprobe with --ignore-install to cause the normal action to then take
 place.  Full documentation on this can be found in the modprobe.conf
 and modprobe manual pages.
 8.3. Painfully Slow Or No Failed Link Detection By Miimon
 ---------------------------------------------------------
 	By default, bonding enables the use_carrier option, which
 instructs bonding to trust the driver to maintain carrier state.
 	As discussed in the options section, above, some drivers do
 not support the netif_carrier_on/_off link state tracking system.
 With use_carrier enabled, bonding will always see these links as up,
 regardless of their actual state.
 	Additionally, other drivers do support netif_carrier, but do
 not maintain it in real time, e.g., only polling the link state at
 some fixed interval.  In this case, miimon will detect failures, but
 only after some long period of time has expired.  If it appears that
 miimon is very slow in detecting link failures, try specifying
 use_carrier=0 to see if that improves the failure detection time.  If
 it does, then it may be that the driver checks the carrier state at a
 fixed interval, but does not cache the MII register values (so the
 use_carrier=0 method of querying the registers directly works).  If
 use_carrier=0 does not improve the failover, then the driver may cache
 the registers, or the problem may be elsewhere.
 	Also, remember that miimon only checks for the device's
 carrier state.  It has no way to determine the state of devices on or
 beyond other ports of a switch, or if a switch is refusing to pass
 traffic while still maintaining carrier on.
 9. SNMP agents
 ===============
 	If running SNMP agents, the bonding driver should be loaded
 before any network drivers participating in a bond.  This requirement
 is due to the interface index (ipAdEntIfIndex) being associated to
 the first interface found with a given IP address.  That is, there is
 only one ipAdEntIfIndex for each IP address.  For example, if eth0 and
 eth1 are slaves of bond0 and the driver for eth0 is loaded before the
 bonding driver, the interface for the IP address will be associated
 with the eth0 interface.  This configuration is shown below, the IP
 address 192.168.1.1 has an interface index of 2 which indexes to eth0
 in the ifDescr table (ifDescr.2).
     interfaces.ifTable.ifEntry.ifDescr.1 = lo
     interfaces.ifTable.ifEntry.ifDescr.2 = eth0
     interfaces.ifTable.ifEntry.ifDescr.3 = eth1
     interfaces.ifTable.ifEntry.ifDescr.4 = eth2
     interfaces.ifTable.ifEntry.ifDescr.5 = eth3
     interfaces.ifTable.ifEntry.ifDescr.6 = bond0
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 5
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 4
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1
 	This problem is avoided by loading the bonding driver before
 any network drivers participating in a bond.  Below is an example of
 loading the bonding driver first, the IP address 192.168.1.1 is
 correctly associated with ifDescr.2.
     interfaces.ifTable.ifEntry.ifDescr.1 = lo
     interfaces.ifTable.ifEntry.ifDescr.2 = bond0
     interfaces.ifTable.ifEntry.ifDescr.3 = eth0
     interfaces.ifTable.ifEntry.ifDescr.4 = eth1
     interfaces.ifTable.ifEntry.ifDescr.5 = eth2
     interfaces.ifTable.ifEntry.ifDescr.6 = eth3
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 6
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 5
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1
 	While some distributions may not report the interface name in
 ifDescr, the association between the IP address and IfIndex remains
 and SNMP functions such as Interface_Scan_Next will report that
 association.
 10. Promiscuous mode
 ====================
 	When running network monitoring tools, e.g., tcpdump, it is
 common to enable promiscuous mode on the device, so that all traffic
 is seen (instead of seeing only traffic destined for the local host).
 The bonding driver handles promiscuous mode changes to the bonding
 master device (e.g., bond0), and propagates the setting to the slave
 devices.
 	For the balance-rr, balance-xor, broadcast, and 802.3ad modes,
 the promiscuous mode setting is propagated to all slaves.
 	For the active-backup, balance-tlb and balance-alb modes, the
 promiscuous mode setting is propagated only to the active slave.
 	For balance-tlb mode, the active slave is the slave currently
 receiving inbound traffic.
 	For balance-alb mode, the active slave is the slave used as a
 "primary."  This slave is used for mode-specific control traffic, for
 sending to peers that are unassigned or if the load is unbalanced.
 	For the active-backup, balance-tlb and balance-alb modes, when
 the active slave changes (e.g., due to a link failure), the
 promiscuous setting will be propagated to the new active slave.
 11. Configuring Bonding for High Availability
 =============================================
 	High Availability refers to configurations that provide
 maximum network availability by having redundant or backup devices,
 links or switches between the host and the rest of the world.  The
 goal is to provide the maximum availability of network connectivity
 (i.e., the network always works), even though other configurations
 could provide higher throughput.
 11.1 High Availability in a Single Switch Topology
 --------------------------------------------------
 	If two hosts (or a host and a single switch) are directly
 connected via multiple physical links, then there is no availability
 penalty to optimizing for maximum bandwidth.  In this case, there is
 only one switch (or peer), so if it fails, there is no alternative
 access to fail over to.  Additionally, the bonding load balance modes
 support link monitoring of their members, so if individual links fail,
 the load will be rebalanced across the remaining devices.
 	See Section 13, "Configuring Bonding for Maximum Throughput"
 for information on configuring bonding with one peer device.
 11.2 High Availability in a Multiple Switch Topology
 ----------------------------------------------------
 	With multiple switches, the configuration of bonding and the
 network changes dramatically.  In multiple switch topologies, there is
 a trade off between network availability and usable bandwidth.
 	Below is a sample network, configured to maximize the
 availability of the network:
                |                                     |
                |port3                           port3|
          +-----+----+                          +-----+----+
          |          |port2       ISL      port2|          |
          | switch A +--------------------------+ switch B |
          |          |                          |          |
          +-----+----+                          +-----++---+
                |port1                           port1|
                |             +-------+               |
                +-------------+ host1 +---------------+
                         eth0 +-------+ eth1
 	In this configuration, there is a link between the two
 switches (ISL, or inter switch link), and multiple ports connecting to
 the outside world ("port3" on each switch).  There is no technical
 reason that this could not be extended to a third switch.
 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
 -------------------------------------------------------------
 	In a topology such as the example above, the active-backup and
 broadcast modes are the only useful bonding modes when optimizing for
 availability; the other modes require all links to terminate on the
 same peer for them to behave rationally.
 active-backup: This is generally the preferred mode, particularly if
 	the switches have an ISL and play together well.  If the
 	network configuration is such that one switch is specifically
 	a backup switch (e.g., has lower capacity, higher cost, etc),
 	then the primary option can be used to insure that the
 	preferred link is always used when it is available.
 broadcast: This mode is really a special purpose mode, and is suitable
 	only for very specific needs.  For example, if the two
 	switches are not connected (no ISL), and the networks beyond
 	them are totally independent.  In this case, if it is
 	necessary for some specific one-way traffic to reach both
 	independent networks, then the broadcast mode may be suitable.
 11.2.2 HA Link Monitoring Selection for Multiple Switch Topology
 ----------------------------------------------------------------
 	The choice of link monitoring ultimately depends upon your
 switch.  If the switch can reliably fail ports in response to other
 failures, then either the MII or ARP monitors should work.  For
 example, in the above example, if the "port3" link fails at the remote
 end, the MII monitor has no direct means to detect this.  The ARP
 monitor could be configured with a target at the remote end of port3,
 thus detecting that failure without switch support.
 	In general, however, in a multiple switch topology, the ARP
 monitor can provide a higher level of reliability in detecting end to
 end connectivity failures (which may be caused by the failure of any
 individual component to pass traffic for any reason).  Additionally,
 the ARP monitor should be configured with multiple targets (at least
 one for each switch in the network).  This will insure that,
 regardless of which switch is active, the ARP monitor has a suitable
 target to query.
 12. Configuring Bonding for Maximum Throughput
 ==============================================
 12.1 Maximizing Throughput in a Single Switch Topology
 ------------------------------------------------------
 	In a single switch configuration, the best method to maximize
 throughput depends upon the application and network environment.  The
 various load balancing modes each have strengths and weaknesses in
 different environments, as detailed below.
 	For this discussion, we will break down the topologies into
 two categories.  Depending upon the destination of most traffic, we
 categorize them into either "gatewayed" or "local" configurations.
 	In a gatewayed configuration, the "switch" is acting primarily
 as a router, and the majority of traffic passes through this router to
 other networks.  An example would be the following:
     +----------+                     +----------+
     |          |eth0            port1|          | to other networks
     | Host A   +---------------------+ router   +------------------->
     |          +---------------------+          | Hosts B and C are out
     |          |eth1            port2|          | here somewhere
     +----------+                     +----------+
 	The router may be a dedicated router device, or another host
 acting as a gateway.  For our discussion, the important point is that
 the majority of traffic from Host A will pass through the router to
 some other network before reaching its final destination.
 	In a gatewayed network configuration, although Host A may
 communicate with many other systems, all of its traffic will be sent
 and received via one other peer on the local network, the router.
 	Note that the case of two systems connected directly via
 multiple physical links is, for purposes of configuring bonding, the
 same as a gatewayed configuration.  In that case, it happens that all
 traffic is destined for the "gateway" itself, not some other network
 beyond the gateway.
 	In a local configuration, the "switch" is acting primarily as
 a switch, and the majority of traffic passes through this switch to
 reach other stations on the same network.  An example would be the
 following:
    +----------+            +----------+       +--------+
    |          |eth0   port1|          +-------+ Host B |
    |  Host A  +------------+  switch  |port3  +--------+
    |          +------------+          |                  +--------+
    |          |eth1   port2|          +------------------+ Host C |
    +----------+            +----------+port4             +--------+
 	Again, the switch may be a dedicated switch device, or another
 host acting as a gateway.  For our discussion, the important point is
 that the majority of traffic from Host A is destined for other hosts
 on the same local network (Hosts B and C in the above example).
 	In summary, in a gatewayed configuration, traffic to and from
 the bonded device will be to the same MAC level peer on the network
 (the gateway itself, i.e., the router), regardless of its final
 destination.  In a local configuration, traffic flows directly to and
 from the final destinations, thus, each destination (Host B, Host C)
 will be addressed directly by their individual MAC addresses.
 	This distinction between a gatewayed and a local network
 configuration is important because many of the load balancing modes
 available use the MAC addresses of the local network source and
 destination to make load balancing decisions.  The behavior of each
 mode is described below.
 12.1.1 MT Bonding Mode Selection for Single Switch Topology
 -----------------------------------------------------------
 	This configuration is the easiest to set up and to understand,
 although you will have to decide which bonding mode best suits your
 needs.  The trade offs for each mode are detailed below:
 balance-rr: This mode is the only mode that will permit a single
 	TCP/IP connection to stripe traffic across multiple
 	interfaces. It is therefore the only mode that will allow a
 	single TCP/IP stream to utilize more than one interface's
 	worth of throughput.  This comes at a cost, however: the
 	striping often results in peer systems receiving packets out
 	of order, causing TCP/IP's congestion control system to kick
 	in, often by retransmitting segments.
 	It is possible to adjust TCP/IP's congestion limits by
 	altering the net.ipv4.tcp_reordering sysctl parameter.  The
 	usual default value is 3, and the maximum useful value is 127.
 	For a four interface balance-rr bond, expect that a single
 	TCP/IP stream will utilize no more than approximately 2.3
 	interface's worth of throughput, even after adjusting
 	tcp_reordering.
 	Note that this out of order delivery occurs when both the
 	sending and receiving systems are utilizing a multiple
 	interface bond.  Consider a configuration in which a
 	balance-rr bond feeds into a single higher capacity network
 	channel (e.g., multiple 100Mb/sec ethernets feeding a single
 	gigabit ethernet via an etherchannel capable switch).  In this
 	configuration, traffic sent from the multiple 100Mb devices to
 	a destination connected to the gigabit device will not see
 	packets out of order.  However, traffic sent from the gigabit
 	device to the multiple 100Mb devices may or may not see
 	traffic out of order, depending upon the balance policy of the
 	switch.  Many switches do not support any modes that stripe
 	traffic (instead choosing a port based upon IP or MAC level
 	addresses); for those devices, traffic flowing from the
 	gigabit device to the many 100Mb devices will only utilize one
 	interface.
 	If you are utilizing protocols other than TCP/IP, UDP for
 	example, and your application can tolerate out of order
 	delivery, then this mode can allow for single stream datagram
 	performance that scales near linearly as interfaces are added
 	to the bond.
 	This mode requires the switch to have the appropriate ports
 	configured for "etherchannel" or "trunking."
 active-backup: There is not much advantage in this network topology to
 	the active-backup mode, as the inactive backup devices are all
 	connected to the same peer as the primary.  In this case, a
 	load balancing mode (with link monitoring) will provide the
 	same level of network availability, but with increased
 	available bandwidth.  On the plus side, active-backup mode
 	does not require any configuration of the switch, so it may
 	have value if the hardware available does not support any of
 	the load balance modes.
 balance-xor: This mode will limit traffic such that packets destined
 	for specific peers will always be sent over the same
 	interface.  Since the destination is determined by the MAC
 	addresses involved, this mode works best in a "local" network
 	configuration (as described above), with destinations all on
 	the same local network.  This mode is likely to be suboptimal
 	if all your traffic is passed through a single router (i.e., a
 	"gatewayed" network configuration, as described above).
 	As with balance-rr, the switch ports need to be configured for
 	"etherchannel" or "trunking."
 broadcast: Like active-backup, there is not much advantage to this
 	mode in this type of network topology.
 802.3ad: This mode can be a good choice for this type of network
 	topology.  The 802.3ad mode is an IEEE standard, so all peers
 	that implement 802.3ad should interoperate well.  The 802.3ad
 	protocol includes automatic configuration of the aggregates,
 	so minimal manual configuration of the switch is needed
 	(typically only to designate that some set of devices is
 	available for 802.3ad).  The 802.3ad standard also mandates
 	that frames be delivered in order (within certain limits), so
 	in general single connections will not see misordering of
 	packets.  The 802.3ad mode does have some drawbacks: the
 	standard mandates that all devices in the aggregate operate at
 	the same speed and duplex.  Also, as with all bonding load
 	balance modes other than balance-rr, no single connection will
 	be able to utilize more than a single interface's worth of
 	bandwidth.  
 	Additionally, the linux bonding 802.3ad implementation
 	distributes traffic by peer (using an XOR of MAC addresses),
 	so in a "gatewayed" configuration, all outgoing traffic will
 	generally use the same device.  Incoming traffic may also end
 	up on a single device, but that is dependent upon the
 	balancing policy of the peer's 8023.ad implementation.  In a
 	"local" configuration, traffic will be distributed across the
 	devices in the bond.
 	Finally, the 802.3ad mode mandates the use of the MII monitor,
 	therefore, the ARP monitor is not available in this mode.
 balance-tlb: The balance-tlb mode balances outgoing traffic by peer.
 	Since the balancing is done according to MAC address, in a
 	"gatewayed" configuration (as described above), this mode will
 	send all traffic across a single device.  However, in a
 	"local" network configuration, this mode balances multiple
 	local network peers across devices in a vaguely intelligent
 	manner (not a simple XOR as in balance-xor or 802.3ad mode),
 	so that mathematically unlucky MAC addresses (i.e., ones that
 	XOR to the same value) will not all "bunch up" on a single
 	interface.
 	Unlike 802.3ad, interfaces may be of differing speeds, and no
 	special switch configuration is required.  On the down side,
 	in this mode all incoming traffic arrives over a single
 	interface, this mode requires certain ethtool support in the
 	network device driver of the slave interfaces, and the ARP
 	monitor is not available.
 balance-alb: This mode is everything that balance-tlb is, and more.
 	It has all of the features (and restrictions) of balance-tlb,
 	and will also balance incoming traffic from local network
 	peers (as described in the Bonding Module Options section,
 	above).
 	The only additional down side to this mode is that the network
 	device driver must support changing the hardware address while
 	the device is open.
 12.1.2 MT Link Monitoring for Single Switch Topology
 ----------------------------------------------------
 	The choice of link monitoring may largely depend upon which
 mode you choose to use.  The more advanced load balancing modes do not
 support the use of the ARP monitor, and are thus restricted to using
 the MII monitor (which does not provide as high a level of end to end
 assurance as the ARP monitor).
 12.2 Maximum Throughput in a Multiple Switch Topology
 -----------------------------------------------------
 	Multiple switches may be utilized to optimize for throughput
 when they are configured in parallel as part of an isolated network
 between two or more systems, for example:
                       +-----------+
                       |  Host A   | 
                       +-+---+---+-+
                         |   |   |
                +--------+   |   +---------+
                |            |             |
         +------+---+  +-----+----+  +-----+----+
         | Switch A |  | Switch B |  | Switch C |
         +------+---+  +-----+----+  +-----+----+
                |            |             |
                +--------+   |   +---------+
                         |   |   |
                       +-+---+---+-+
                       |  Host B   | 
                       +-----------+
 	In this configuration, the switches are isolated from one
 another.  One reason to employ a topology such as this is for an
 isolated network with many hosts (a cluster configured for high
 performance, for example), using multiple smaller switches can be more
 cost effective than a single larger switch, e.g., on a network with 24
 hosts, three 24 port switches can be significantly less expensive than
 a single 72 port switch.
 	If access beyond the network is required, an individual host
 can be equipped with an additional network device connected to an
 external network; this host then additionally acts as a gateway.
 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
 -------------------------------------------------------------
 	In actual practice, the bonding mode typically employed in
 configurations of this type is balance-rr.  Historically, in this
 network configuration, the usual caveats about out of order packet
 delivery are mitigated by the use of network adapters that do not do
 any kind of packet coalescing (via the use of NAPI, or because the
 device itself does not generate interrupts until some number of
 packets has arrived).  When employed in this fashion, the balance-rr
 mode allows individual connections between two hosts to effectively
 utilize greater than one interface's bandwidth.
 12.2.2 MT Link Monitoring for Multiple Switch Topology
 ------------------------------------------------------
 	Again, in actual practice, the MII monitor is most often used
 in this configuration, as performance is given preference over
 availability.  The ARP monitor will function in this topology, but its
 advantages over the MII monitor are mitigated by the volume of probes
 needed as the number of systems involved grows (remember that each
 host in the network is configured with bonding).
 13. Switch Behavior Issues
 ==========================
 13.1 Link Establishment and Failover Delays
 -------------------------------------------
 	Some switches exhibit undesirable behavior with regard to the
 timing of link up and down reporting by the switch.
 	First, when a link comes up, some switches may indicate that
 the link is up (carrier available), but not pass traffic over the
 interface for some period of time.  This delay is typically due to
 some type of autonegotiation or routing protocol, but may also occur
 during switch initialization (e.g., during recovery after a switch
 failure).  If you find this to be a problem, specify an appropriate
 value to the updelay bonding module option to delay the use of the
 relevant interface(s).
 	Second, some switches may "bounce" the link state one or more
 times while a link is changing state.  This occurs most commonly while
 the switch is initializing.  Again, an appropriate updelay value may
 help.
 	Note that when a bonding interface has no active links, the
 driver will immediately reuse the first link that goes up, even if the
 updelay parameter has been specified (the updelay is ignored in this
 case).  If there are slave interfaces waiting for the updelay timeout
 to expire, the interface that first went into that state will be
 immediately reused.  This reduces down time of the network if the
 value of updelay has been overestimated, and since this occurs only in
 cases with no connectivity, there is no additional penalty for
 ignoring the updelay.
 	In addition to the concerns about switch timings, if your
 switches take a long time to go into backup mode, it may be desirable
 to not activate a backup interface immediately after a link goes down.
 Failover may be delayed via the downdelay bonding module option.
 13.2 Duplicated Incoming Packets
 --------------------------------
 	It is not uncommon to observe a short burst of duplicated
 traffic when the bonding device is first used, or after it has been
 idle for some period of time.  This is most easily observed by issuing
 a "ping" to some other host on the network, and noticing that the
 output from ping flags duplicates (typically one per slave).
 	For example, on a bond in active-backup mode with five slaves
 all connected to one switch, the output may appear as follows:
 # ping -n 10.0.4.2
 PING 10.0.4.2 (10.0.4.2) from 10.0.3.10 : 56(84) bytes of data.
 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.7 ms
 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
 64 bytes from 10.0.4.2: icmp_seq=1 ttl=64 time=13.8 ms (DUP!)
 64 bytes from 10.0.4.2: icmp_seq=2 ttl=64 time=0.216 ms
 64 bytes from 10.0.4.2: icmp_seq=3 ttl=64 time=0.267 ms
 64 bytes from 10.0.4.2: icmp_seq=4 ttl=64 time=0.222 ms
 	This is not due to an error in the bonding driver, rather, it
 is a side effect of how many switches update their MAC forwarding
 tables.  Initially, the switch does not associate the MAC address in
 the packet with a particular switch port, and so it may send the
 traffic to all ports until its MAC forwarding table is updated.  Since
 the interfaces attached to the bond may occupy multiple ports on a
 single switch, when the switch (temporarily) floods the traffic to all
 ports, the bond device receives multiple copies of the same packet
 (one per slave device).
 	The duplicated packet behavior is switch dependent, some
 switches exhibit this, and some do not.  On switches that display this
 behavior, it can be induced by clearing the MAC forwarding table (on
 most Cisco switches, the privileged command "clear mac address-table
 dynamic" will accomplish this).
 14. Hardware Specific Considerations
 ====================================
 	This section contains additional information for configuring
 bonding on specific hardware platforms, or for interfacing bonding
 with particular switches or other devices.
 14.1 IBM BladeCenter
 --------------------
 	This applies to the JS20 and similar systems.
 	On the JS20 blades, the bonding driver supports only
 balance-rr, active-backup, balance-tlb and balance-alb modes.  This is
 largely due to the network topology inside the BladeCenter, detailed
 below.
 JS20 network adapter information
 --------------------------------
 	All JS20s come with two Broadcom Gigabit Ethernet ports
 integrated on the planar (that's "motherboard" in IBM-speak).  In the
 BladeCenter chassis, the eth0 port of all JS20 blades is hard wired to
 I/O Module #1; similarly, all eth1 ports are wired to I/O Module #2.
 An add-on Broadcom daughter card can be installed on a JS20 to provide
 two more Gigabit Ethernet ports.  These ports, eth2 and eth3, are
 wired to I/O Modules 3 and 4, respectively.
 	Each I/O Module may contain either a switch or a passthrough
 module (which allows ports to be directly connected to an external
 switch).  Some bonding modes require a specific BladeCenter internal
 network topology in order to function; these are detailed below.
 	Additional BladeCenter-specific networking information can be
 found in two IBM Redbooks (www.ibm.com/redbooks):
 "IBM eServer BladeCenter Networking Options"
 "IBM eServer BladeCenter Layer 2-7 Network Switching"
 BladeCenter networking configuration
 ------------------------------------
 	Because a BladeCenter can be configured in a very large number
 of ways, this discussion will be confined to describing basic
 configurations.
 	Normally, Ethernet Switch Modules (ESMs) are used in I/O
 modules 1 and 2.  In this configuration, the eth0 and eth1 ports of a
 JS20 will be connected to different internal switches (in the
 respective I/O modules).
 	A passthrough module (OPM or CPM, optical or copper,
 passthrough module) connects the I/O module directly to an external
 switch.  By using PMs in I/O module #1 and #2, the eth0 and eth1
 interfaces of a JS20 can be redirected to the outside world and
 connected to a common external switch.
 	Depending upon the mix of ESMs and PMs, the network will
 appear to bonding as either a single switch topology (all PMs) or as a
 multiple switch topology (one or more ESMs, zero or more PMs).  It is
 also possible to connect ESMs together, resulting in a configuration
 much like the example in "High Availability in a Multiple Switch
 Topology," above.
 Requirements for specific modes
 -------------------------------
 	The balance-rr mode requires the use of passthrough modules
 for devices in the bond, all connected to an common external switch.
 That switch must be configured for "etherchannel" or "trunking" on the
 appropriate ports, as is usual for balance-rr.
 	The balance-alb and balance-tlb modes will function with
 either switch modules or passthrough modules (or a mix).  The only
 specific requirement for these modes is that all network interfaces
 must be able to reach all destinations for traffic sent over the
 bonding device (i.e., the network must converge at some point outside
 the BladeCenter).
 	The active-backup mode has no additional requirements.
 Link monitoring issues
 ----------------------
 	When an Ethernet Switch Module is in place, only the ARP
 monitor will reliably detect link loss to an external switch.  This is
 nothing unusual, but examination of the BladeCenter cabinet would
 suggest that the "external" network ports are the ethernet ports for
 the system, when it fact there is a switch between these "external"
 ports and the devices on the JS20 system itself.  The MII monitor is
 only able to detect link failures between the ESM and the JS20 system.
 	When a passthrough module is in place, the MII monitor does
 detect failures to the "external" port, which is then directly
 connected to the JS20 system.
 Other concerns
 --------------
 	The Serial Over LAN (SoL) link is established over the primary
 ethernet (eth0) only, therefore, any loss of link to eth0 will result
 in losing your SoL connection.  It will not fail over with other
 network traffic, as the SoL system is beyond the control of the
 bonding driver.
 	It may be desirable to disable spanning tree on the switch
 (either the internal Ethernet Switch Module, or an external switch) to
 avoid fail-over delay issues when using bonding.
 15. Frequently Asked Questions
 ==============================
 1.  Is it SMP safe?
 	Yes. The old 2.0.xx channel bonding patch was not SMP safe.
 The new driver was designed to be SMP safe from the start.
 2.  What type of cards will work with it?
 	Any Ethernet type cards (you can even mix cards - a Intel
 EtherExpress PRO/100 and a 3com 3c905b, for example).  For most modes,
 devices need not be of the same speed.
 3.  How many bonding devices can I have?
 	There is no limit.
 4.  How many slaves can a bonding device have?
 	This is limited only by the number of network interfaces Linux
 supports and/or the number of network cards you can place in your
 system.
 5.  What happens when a slave link dies?
 	If link monitoring is enabled, then the failing device will be
 disabled.  The active-backup mode will fail over to a backup link, and
 other modes will ignore the failed link.  The link will continue to be
 monitored, and should it recover, it will rejoin the bond (in whatever
 manner is appropriate for the mode). See the sections on High
 Availability and the documentation for each mode for additional
 information.
 	Link monitoring can be enabled via either the miimon or
 arp_interval parameters (described in the module parameters section,
 above).  In general, miimon monitors the carrier state as sensed by
 the underlying network device, and the arp monitor (arp_interval)
 monitors connectivity to another host on the local network.
 	If no link monitoring is configured, the bonding driver will
 be unable to detect link failures, and will assume that all links are
 always available.  This will likely result in lost packets, and a
 resulting degradation of performance.  The precise performance loss
 depends upon the bonding mode and network configuration.
 6.  Can bonding be used for High Availability?
 	Yes.  See the section on High Availability for details.
 7.  Which switches/systems does it work with?
 	The full answer to this depends upon the desired mode.
 	In the basic balance modes (balance-rr and balance-xor), it
 works with any system that supports etherchannel (also called
 trunking).  Most managed switches currently available have such
 support, and many unmanaged switches as well.
 	The advanced balance modes (balance-tlb and balance-alb) do
 not have special switch requirements, but do need device drivers that
 support specific features (described in the appropriate section under
 module parameters, above).
 	In 802.3ad mode, it works with systems that support IEEE
 802.3ad Dynamic Link Aggregation.  Most managed and many unmanaged
 switches currently available support 802.3ad.
        The active-backup mode should work with any Layer-II switch.
 8.  Where does a bonding device get its MAC address from?
 	If not explicitly configured (with ifconfig or ip link), the
 MAC address of the bonding device is taken from its first slave
 device.  This MAC address is then passed to all following slaves and
 remains persistent (even if the first slave is removed) until the
 bonding device is brought down or reconfigured.
 	If you wish to change the MAC address, you can set it with
 ifconfig or ip link:
 # ifconfig bond0 hw ether 00:11:22:33:44:55
 # ip link set bond0 address 66:77:88:99:aa:bb
 	The MAC address can be also changed by bringing down/up the
 device and then changing its slaves (or their order):
 # ifconfig bond0 down ; modprobe -r bonding
 # ifconfig bond0 .... up
 # ifenslave bond0 eth...
 	This method will automatically take the address from the next
 slave that is added.
 	To restore your slaves' MAC addresses, you need to detach them
 from the bond (`ifenslave -d bond0 eth0'). The bonding driver will
 then restore the MAC addresses that the slaves had before they were
 enslaved.
 16. Resources and Links
 =======================
 The latest version of the bonding driver can be found in the latest
 version of the linux kernel, found on http://kernel.org
 The latest version of this document can be found in either the latest
 kernel source (named Documentation/networking/bonding.txt), or on the
 bonding sourceforge site:
 http://www.sourceforge.net/projects/bonding
 Discussions regarding the bonding driver take place primarily on the
 bonding-devel mailing list, hosted at sourceforge.net.  If you have
 questions or problems, post them to the list.  The list address is:
 bonding-devel@lists.sourceforge.net
 	The administrative interface (to subscribe or unsubscribe) can
 be found at:
 https://lists.sourceforge.net/lists/listinfo/bonding-devel
 Donald Becker's Ethernet Drivers and diag programs may be found at :
 - http://www.scyld.com/network/
 You will also find a lot of information regarding Ethernet, NWay, MII,
 etc. at www.scyld.com.
 -- END --
 NOTE
 ----
 This document was contributed by Cirrus Logic for kernel 2.2.5.  This version
 has been updated for 2.3.48 by Andrew Morton <andrewm@uow.edu.au>
 Cirrus make a copy of this driver available at their website, as
 described below.  In general, you should use the driver version which
 comes with your Linux distribution.
 CIRRUS LOGIC LAN CS8900/CS8920 ETHERNET ADAPTERS
 Linux Network Interface Driver ver. 2.00 <kernel 2.3.48>
 ===============================================================================
 TABLE OF CONTENTS
 1.0 CIRRUS LOGIC LAN CS8900/CS8920 ETHERNET ADAPTERS
    1.1 Product Overview 
    1.2 Driver Description
 	1.2.1 Driver Name
 	1.2.2 File in the Driver Package
    1.3 System Requirements
    1.4 Licensing Information
 2.0 ADAPTER INSTALLATION and CONFIGURATION
    2.1 CS8900-based Adapter Configuration
    2.2 CS8920-based Adapter Configuration 
 3.0 LOADING THE DRIVER AS A MODULE
 4.0 COMPILING THE DRIVER
    4.1 Compiling the Driver as a Loadable Module
    4.2 Compiling the driver to support memory mode
    4.3 Compiling the driver to support Rx DMA 
    4.4 Compiling the Driver into the Kernel
 5.0 TESTING AND TROUBLESHOOTING
    5.1 Known Defects and Limitations
    5.2 Testing the Adapter
        5.2.1 Diagnostic Self-Test
        5.2.2 Diagnostic Network Test
    5.3 Using the Adapter's LEDs
    5.4 Resolving I/O Conflicts
 6.0 TECHNICAL SUPPORT
    6.1 Contacting Cirrus Logic's Technical Support
    6.2 Information Required Before Contacting Technical Support
    6.3 Obtaining the Latest Driver Version
    6.4 Current maintainer
    6.5 Kernel boot parameters
 1.0 CIRRUS LOGIC LAN CS8900/CS8920 ETHERNET ADAPTERS
 ===============================================================================
 1.1 PRODUCT OVERVIEW
 The CS8900-based ISA Ethernet Adapters from Cirrus Logic follow 
 IEEE 802.3 standards and support half or full-duplex operation in ISA bus 
 computers on 10 Mbps Ethernet networks.  The adapters are designed for operation 
 in 16-bit ISA or EISA bus expansion slots and are available in 
 10BaseT-only or 3-media configurations (10BaseT, 10Base2, and AUI for 10Base-5 
 or fiber networks).  
 CS8920-based adapters are similar to the CS8900-based adapter with additional 
 features for Plug and Play (PnP) support and Wakeup Frame recognition.  As 
 such, the configuration procedures differ somewhat between the two types of 
 adapters.  Refer to the "Adapter Configuration" section for details on 
 configuring both types of adapters.
 1.2 DRIVER DESCRIPTION
 The CS8900/CS8920 Ethernet Adapter driver for Linux supports the Linux
 v2.3.48 or greater kernel.  It can be compiled directly into the kernel
 or loaded at run-time as a device driver module.
 1.2.1 Driver Name: cs89x0
 1.2.2 Files in the Driver Archive:
 The files in the driver at Cirrus' website include:
  readme.txt         - this file
  build              - batch file to compile cs89x0.c.
  cs89x0.c           - driver C code
  cs89x0.h           - driver header file
  cs89x0.o           - pre-compiled module (for v2.2.5 kernel)
  config/Config.in   - sample file to include cs89x0 driver in the kernel.
  config/Makefile    - sample file to include cs89x0 driver in the kernel.
  config/Space.c     - sample file to include cs89x0 driver in the kernel.
 1.3 SYSTEM REQUIREMENTS
 The following hardware is required:
   * Cirrus Logic LAN (CS8900/20-based) Ethernet ISA Adapter   
   * IBM or IBM-compatible PC with:
     * An 80386 or higher processor
     * 16 bytes of contiguous IO space available between 210h - 370h
     * One available IRQ (5,10,11,or 12 for the CS8900, 3-7,9-15 for CS8920).
   * Appropriate cable (and connector for AUI, 10BASE-2) for your network
     topology.
 The following software is required:
 * LINUX kernel version 2.3.48 or higher
   * CS8900/20 Setup Utility (DOS-based)
   * LINUX kernel sources for your kernel (if compiling into kernel)
   * GNU Toolkit (gcc and make) v2.6 or above (if compiling into kernel 
     or a module)   
 1.4 LICENSING INFORMATION
 This program is free software; you can redistribute it and/or modify it under
 the terms of the GNU General Public License as published by the Free Software
 Foundation, version 1.
 This program is distributed in the hope that it will be useful, but WITHOUT
 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 
 FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for 
 more details.
 For a full copy of the GNU General Public License, write to the Free Software
 Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 2.0 ADAPTER INSTALLATION and CONFIGURATION
 ===============================================================================
 Both the CS8900 and CS8920-based adapters can be configured using parameters 
 stored in an on-board EEPROM. You must use the DOS-based CS8900/20 Setup 
 Utility if you want to change the adapter's configuration in EEPROM.  
 When loading the driver as a module, you can specify many of the adapter's 
 configuration parameters on the command-line to override the EEPROM's settings 
 or for interface configuration when an EEPROM is not used. (CS8920-based 
 adapters must use an EEPROM.) See Section 3.0 LOADING THE DRIVER AS A MODULE.
 Since the CS8900/20 Setup Utility is a DOS-based application, you must install 
 and configure the adapter in a DOS-based system using the CS8900/20 Setup 
 Utility before installation in the target LINUX system.  (Not required if 
 installing a CS8900-based adapter and the default configuration is acceptable.)
 2.1 CS8900-BASED ADAPTER CONFIGURATION
 CS8900-based adapters shipped from Cirrus Logic have been configured 
 with the following "default" settings:
  Operation Mode:      Memory Mode
  IRQ:                 10
  Base I/O Address:    300
  Memory Base Address: D0000
  Optimization:	       DOS Client
  Transmission Mode:   Half-duplex
  BootProm:            None
  Media Type:	       Autodetect (3-media cards) or 
                       10BASE-T (10BASE-T only adapter)
 You should only change the default configuration settings if conflicts with 
 another adapter exists. To change the adapter's configuration, run the 
 CS8900/20 Setup Utility. 
 2.2 CS8920-BASED ADAPTER CONFIGURATION
 CS8920-based adapters are shipped from Cirrus Logic configured as Plug
 and Play (PnP) enabled.  However, since the cs89x0 driver does NOT
 support PnP, you must install the CS8920 adapter in a DOS-based PC and
 run the CS8900/20 Setup Utility to disable PnP and configure the
 adapter before installation in the target Linux system.  Failure to do
 this will leave the adapter inactive and the driver will be unable to
 communicate with the adapter.  
        **************************************************************** 
        *                    CS8920-BASED ADAPTERS:                    *
        *                                                              * 
        * CS8920-BASED ADAPTERS ARE PLUG and PLAY ENABLED BY DEFAULT.  * 
        * THE CS89X0 DRIVER DOES NOT SUPPORT PnP. THEREFORE, YOU MUST  *
        * RUN THE CS8900/20 SETUP UTILITY TO DISABLE PnP SUPPORT AND   *
        * TO ACTIVATE THE ADAPTER.                                     *
        ****************************************************************
 3.0 LOADING THE DRIVER AS A MODULE
 ===============================================================================
 If the driver is compiled as a loadable module, you can load the driver module
 with the 'modprobe' command.  Many of the adapter's configuration parameters can 
 be specified as command-line arguments to the load command.  This facility 
 provides a means to override the EEPROM's settings or for interface 
 configuration when an EEPROM is not used.
 Example:
    insmod cs89x0.o io=0x200 irq=0xA media=aui
 This example loads the module and configures the adapter to use an IO port base
 address of 200h, interrupt 10, and use the AUI media connection.  The following
 configuration options are available on the command line:
 * io=###               - specify IO address (200h-360h)
 * irq=##               - specify interrupt level
 * use_dma=1            - Enable DMA
 * dma=#                - specify dma channel (Driver is compiled to support
                         Rx DMA only)
 * dmasize=# (16 or 64) - DMA size 16K or 64K.  Default value is set to 16.
 * media=rj45           - specify media type
   or media=bnc
   or media=aui
   or media=auto
 * duplex=full          - specify forced half/full/autonegotiate duplex
   or duplex=half
   or duplex=auto
 * debug=#              - debug level (only available if the driver was compiled
                         for debugging)
 NOTES:
 a) If an EEPROM is present, any specified command-line parameter
   will override the corresponding configuration value stored in
   EEPROM.
 b) The "io" parameter must be specified on the command-line.  
 c) The driver's hardware probe routine is designed to avoid
   writing to I/O space until it knows that there is a cs89x0
   card at the written addresses.  This could cause problems
   with device probing.  To avoid this behaviour, add one
   to the `io=' module parameter.  This doesn't actually change
   the I/O address, but it is a flag to tell the driver
   topartially initialise the hardware before trying to
   identify the card.  This could be dangerous if you are
   not sure that there is a cs89x0 card at the provided address.
   For example, to scan for an adapter located at IO base 0x300,
   specify an IO address of 0x301.  
 d) The "duplex=auto" parameter is only supported for the CS8920.
 e) The minimum command-line configuration required if an EEPROM is
   not present is:
   io 
   irq 
   media type (no autodetect)
 f) The following additional parameters are CS89XX defaults (values
   used with no EEPROM or command-line argument).
   * DMA Burst = enabled
   * IOCHRDY Enabled = enabled
   * UseSA = enabled
   * CS8900 defaults to half-duplex if not specified on command-line
   * CS8920 defaults to autoneg if not specified on command-line
   * Use reset defaults for other config parameters
   * dma_mode = 0
 g) You can use ifconfig to set the adapter's Ethernet address.
 h) Many Linux distributions use the 'modprobe' command to load
   modules.  This program uses the '/etc/conf.modules' file to
   determine configuration information which is passed to a driver
   module when it is loaded.  All the configuration options which are
   described above may be placed within /etc/conf.modules.
   For example:
   > cat /etc/conf.modules
   ...
   alias eth0 cs89x0
   options cs89x0 io=0x0200 dma=5 use_dma=1
   ...
   In this example we are telling the module system that the
   ethernet driver for this machine should use the cs89x0 driver.  We
   are asking 'modprobe' to pass the 'io', 'dma' and 'use_dma'
   arguments to the driver when it is loaded.
 i) Cirrus recommend that the cs89x0 use the ISA DMA channels 5, 6 or
   7.  You will probably find that other DMA channels will not work.
 j) The cs89x0 supports DMA for receiving only.  DMA mode is
   significantly more efficient.  Flooding a 400 MHz Celeron machine
   with large ping packets consumes 82% of its CPU capacity in non-DMA
   mode.  With DMA this is reduced to 45%.
 k) If your Linux kernel was compiled with inbuilt plug-and-play
   support you will be able to find information about the cs89x0 card
   with the command
   cat /proc/isapnp
 l) If during DMA operation you find erratic behavior or network data
   corruption you should use your PC's BIOS to slow the EISA bus clock.
 m) If the cs89x0 driver is compiled directly into the kernel
   (non-modular) then its I/O address is automatically determined by
   ISA bus probing.  The IRQ number, media options, etc are determined
   from the card's EEPROM.
 n) If the cs89x0 driver is compiled directly into the kernel, DMA
   mode may be selected by providing the kernel with a boot option
   'cs89x0_dma=N' where 'N' is the desired DMA channel number (5, 6 or 7).
   Kernel boot options may be provided on the LILO command line:
 	LILO boot: linux cs89x0_dma=5
   or they may be placed in /etc/lilo.conf:
 	image=/boot/bzImage-2.3.48
 	  append="cs89x0_dma=5"
 	  label=linux
 	  root=/dev/hda5
 	  read-only
   The DMA Rx buffer size is hardwired to 16 kbytes in this mode.
   (64k mode is not available).
 4.0 COMPILING THE DRIVER
 ===============================================================================
 The cs89x0 driver can be compiled directly into the kernel or compiled into
 a loadable device driver module.
 4.1 COMPILING THE DRIVER AS A LOADABLE MODULE
 To compile the driver into a loadable module, use the following command 
 (single command line, without quotes):
 "gcc -D__KERNEL__ -I/usr/src/linux/include -I/usr/src/linux/net/inet -Wall 
 -Wstrict-prototypes -O2 -fomit-frame-pointer -DMODULE -DCONFIG_MODVERSIONS 
 -c cs89x0.c"
 4.2 COMPILING THE DRIVER TO SUPPORT MEMORY MODE
 Support for memory mode was not carried over into the 2.3 series kernels.
 4.3 COMPILING THE DRIVER TO SUPPORT Rx DMA
 The compile-time optionality for DMA was removed in the 2.3 kernel
 series.  DMA support is now unconditionally part of the driver.  It is
 enabled by the 'use_dma=1' module option.
 4.4 COMPILING THE DRIVER INTO THE KERNEL
 If your Linux distribution already has support for the cs89x0 driver
 then simply copy the source file to the /usr/src/linux/drivers/net
 directory to replace the original ones and run the make utility to
 rebuild the kernel.  See Step 3 for rebuilding the kernel.
 If your Linux does not include the cs89x0 driver, you need to edit three 
 configuration files, copy the source file to the /usr/src/linux/drivers/net
 directory, and then run the make utility to rebuild the kernel.
 1. Edit the following configuration files by adding the statements as
 indicated.  (When possible, try to locate the added text to the section of the
 file containing similar statements).
 a.) In /usr/src/linux/drivers/net/Config.in, add:
 tristate 'CS89x0 support' CONFIG_CS89x0
 Example:
     if [ "$CONFIG_EXPERIMENTAL" = "y" ]; then
       tristate 'ICL EtherTeam 16i/32 support' CONFIG_ETH16I
     fi
     tristate 'CS89x0 support' CONFIG_CS89x0
     tristate 'NE2000/NE1000 support' CONFIG_NE2000
     if [ "$CONFIG_EXPERIMENTAL" = "y" ]; then
       tristate 'NI5210 support' CONFIG_NI52
 b.) In /usr/src/linux/drivers/net/Makefile, add the following lines: 
 ifeq ($(CONFIG_CS89x0),y)
 L_OBJS += cs89x0.o
 else
  ifeq ($(CONFIG_CS89x0),m)
  M_OBJS += cs89x0.o
  endif
 endif
 c.) In /linux/drivers/net/Space.c file, add the line:
 extern int cs89x0_probe(struct device *dev);
 Example:
 extern int ultra_probe(struct device *dev);
 extern int wd_probe(struct device *dev);
 extern int el2_probe(struct device *dev);
 extern int cs89x0_probe(struct device *dev);
 extern int ne_probe(struct device *dev);
 extern int hp_probe(struct device *dev);
 extern int hp_plus_probe(struct device *dev);
 Also add:
 #ifdef CONFIG_CS89x0
 	{ cs89x0_probe,0 },
 #endif
 2.) Copy the driver source files (cs89x0.c and cs89x0.h) 
 into the /usr/src/linux/drivers/net directory.
 3.) Go to /usr/src/linux directory and run 'make config' followed by 'make' 
 (or make bzImage) to rebuild the kernel. 
 4.) Use the DOS 'setup' utility to disable plug and play on the NIC.
 5.0 TESTING AND TROUBLESHOOTING
 ===============================================================================
 5.1 KNOWN DEFECTS and LIMITATIONS
 Refer to the RELEASE.TXT file distributed as part of this archive for a list of 
 known defects, driver limitations, and work arounds.
 5.2 TESTING THE ADAPTER
 Once the adapter has been installed and configured, the diagnostic option of 
 the CS8900/20 Setup Utility can be used to test the functionality of the 
 adapter and its network connection.  Use the diagnostics 'Self Test' option to
 test the functionality of the adapter with the hardware configuration you have
 assigned. You can use the diagnostics 'Network Test' to test the ability of the
 adapter to communicate across the Ethernet with another PC equipped with a 
 CS8900/20-based adapter card (it must also be running the CS8900/20 Setup 
 Utility).
         NOTE: The Setup Utility's diagnostics are designed to run in a
         DOS-only operating system environment.  DO NOT run the diagnostics 
         from a DOS or command prompt session under Windows 95, Windows NT, 
         OS/2, or other operating system.
 To run the diagnostics tests on the CS8900/20 adapter:
   1.) Boot DOS on the PC and start the CS8900/20 Setup Utility.
   2.) The adapter's current configuration is displayed.  Hit the ENTER key to
       get to the main menu.
   4.) Select 'Diagnostics' (ALT-G) from the main menu.  
       * Select 'Self-Test' to test the adapter's basic functionality.
       * Select 'Network Test' to test the network connection and cabling.
 5.2.1 DIAGNOSTIC SELF-TEST
 The diagnostic self-test checks the adapter's basic functionality as well as 
 its ability to communicate across the ISA bus based on the system resources 
 assigned during hardware configuration.  The following tests are performed:
   * IO Register Read/Write Test
     The IO Register Read/Write test insures that the CS8900/20 can be 
     accessed in IO mode, and that the IO base address is correct.
   * Shared Memory Test
     The Shared Memory test insures the CS8900/20 can be accessed in memory 
     mode and that the range of memory addresses assigned does not conflict 
     with other devices in the system.
   * Interrupt Test
     The Interrupt test insures there are no conflicts with the assigned IRQ
     signal.
   * EEPROM Test
     The EEPROM test insures the EEPROM can be read.
   * Chip RAM Test
     The Chip RAM test insures the 4K of memory internal to the CS8900/20 is
     working properly.
   * Internal Loop-back Test
     The Internal Loop Back test insures the adapter's transmitter and 
     receiver are operating properly.  If this test fails, make sure the 
     adapter's cable is connected to the network (check for LED activity for 
     example).
   * Boot PROM Test
     The Boot PROM  test insures the Boot PROM is present, and can be read.
     Failure indicates the Boot PROM  was not successfully read due to a
     hardware problem or due to a conflicts on the Boot PROM address
     assignment. (Test only applies if the adapter is configured to use the
     Boot PROM option.)
 Failure of a test item indicates a possible system resource conflict with 
 another device on the ISA bus.  In this case, you should use the Manual Setup 
 option to reconfigure the adapter by selecting a different value for the system
 resource that failed.
 5.2.2 DIAGNOSTIC NETWORK TEST
 The Diagnostic Network Test verifies a working network connection by 
 transferring data between two CS8900/20 adapters installed in different PCs 
 on the same network. (Note: the diagnostic network test should not be run 
 between two nodes across a router.) 
 This test requires that each of the two PCs have a CS8900/20-based adapter
 installed and have the CS8900/20 Setup Utility running.  The first PC is 
 configured as a Responder and the other PC is configured as an Initiator.  
 Once the Initiator is started, it sends data frames to the Responder which 
 returns the frames to the Initiator.
 The total number of frames received and transmitted are displayed on the 
 Initiator's display, along with a count of the number of frames received and 
 transmitted OK or in error.  The test can be terminated anytime by the user at 
 either PC.
 To setup the Diagnostic Network Test:
    1.) Select a PC with a CS8900/20-based adapter and a known working network
        connection to act as the Responder.  Run the CS8900/20 Setup Utility 
        and select 'Diagnostics -> Network Test -> Responder' from the main 
        menu.  Hit ENTER to start the Responder.
    2.) Return to the PC with the CS8900/20-based adapter you want to test and
        start the CS8900/20 Setup Utility. 
    3.) From the main menu, Select 'Diagnostic -> Network Test -> Initiator'.
        Hit ENTER to start the test.
 You may stop the test on the Initiator at any time while allowing the Responder
 to continue running.  In this manner, you can move to additional PCs and test 
 them by starting the Initiator on another PC without having to stop/start the 
 Responder.
 5.3 USING THE ADAPTER'S LEDs
 The 2 and 3-media adapters have two LEDs visible on the back end of the board 
 located near the 10Base-T connector.  
 Link Integrity LED: A "steady" ON of the green LED indicates a valid 10Base-T 
 connection.  (Only applies to 10Base-T.  The green LED has no significance for
 a 10Base-2 or AUI connection.)
 TX/RX LED: The yellow LED lights briefly each time the adapter transmits or 
 receives data. (The yellow LED will appear to "flicker" on a typical network.)
 5.4 RESOLVING I/O CONFLICTS
 An IO conflict occurs when two or more adapter use the same ISA resource (IO 
 address, memory address or IRQ).  You can usually detect an IO conflict in one 
 of four ways after installing and or configuring the CS8900/20-based adapter:
    1.) The system does not boot properly (or at all).
    2.) The driver cannot communicate with the adapter, reporting an "Adapter
        not found" error message.
    3.) You cannot connect to the network or the driver will not load.
    4.) If you have configured the adapter to run in memory mode but the driver
        reports it is using IO mode when loading, this is an indication of a
        memory address conflict.
 If an IO conflict occurs, run the CS8900/20 Setup Utility and perform a 
 diagnostic self-test.  Normally, the ISA resource in conflict will fail the 
 self-test.  If so, reconfigure the adapter selecting another choice for the 
 resource in conflict.  Run the diagnostics again to check for further IO 
 conflicts.
 In some cases, such as when the PC will not boot, it may be necessary to remove
 the adapter and reconfigure it by installing it in another PC to run the 
 CS8900/20 Setup Utility.  Once reinstalled in the target system, run the 
 diagnostics self-test to ensure the new configuration is free of conflicts 
 before loading the driver again.
 When manually configuring the adapter, keep in mind the typical ISA system 
 resource usage as indicated in the tables below.
 I/O Address    	Device                        IRQ      Device
 -----------    	--------                      ---      --------
 200-20F       	Game I/O adapter               3       COM2, Bus Mouse
 230-23F       	Bus Mouse                      4       COM1
 270-27F       	LPT3: third parallel port      5       LPT2
 2F0-2FF       	COM2: second serial port       6       Floppy Disk controller
 320-32F       	Fixed disk controller          7       LPT1
                                      	       8       Real-time Clock
                                                 9       EGA/VGA display adapter    
                                                12       Mouse (PS/2)                              
 Memory Address  Device                          13       Math Coprocessor
 --------------  ---------------------           14       Hard Disk controller
 A000-BFFF	EGA Graphics Adpater
 A000-C7FF	VGA Graphics Adpater
 B000-BFFF	Mono Graphics Adapter
 B800-BFFF	Color Graphics Adapter
 E000-FFFF	AT BIOS
 6.0 TECHNICAL SUPPORT
 ===============================================================================
 6.1 CONTACTING CIRRUS LOGIC'S TECHNICAL SUPPORT
 Cirrus Logic's CS89XX Technical Application Support can be reached at:
 Telephone  :(800) 888-5016 (from inside U.S. and Canada)
           :(512) 442-7555 (from outside the U.S. and Canada)
 Fax        :(512) 912-3871
 Email      :ethernet@crystal.cirrus.com
 WWW        :http://www.cirrus.com
 6.2 INFORMATION REQUIRED BEFORE CONTACTING TECHNICAL SUPPORT
 Before contacting Cirrus Logic for technical support, be prepared to provide as 
 Much of the following information as possible. 
 1.) Adapter type (CRD8900, CDB8900, CDB8920, etc.)
 2.) Adapter configuration
    * IO Base, Memory Base, IO or memory mode enabled, IRQ, DMA channel
    * Plug and Play enabled/disabled (CS8920-based adapters only)
    * Configured for media auto-detect or specific media type (which type).    
 3.) PC System's Configuration
    * Plug and Play system (yes/no)
    * BIOS (make and version)
    * System make and model
    * CPU (type and speed)
    * System RAM
    * SCSI Adapter
 4.) Software
    * CS89XX driver and version
    * Your network operating system and version
    * Your system's OS version 
    * Version of all protocol support files
 5.) Any Error Message displayed.
 6.3 OBTAINING THE LATEST DRIVER VERSION
 You can obtain the latest CS89XX drivers and support software from Cirrus Logic's 
 Web site.  You can also contact Cirrus Logic's Technical Support (email:
 ethernet@crystal.cirrus.com) and request that you be registered for automatic 
 software-update notification.
 Cirrus Logic maintains a web page at http://www.cirrus.com with the
-the latest drivers and technical publications.
+latest drivers and technical publications.
 6.4 Current maintainer
 In February 2000 the maintenance of this driver was assumed by Andrew
 Morton <akpm@zip.com.au>
 6.5 Kernel module parameters
 For use in embedded environments with no cs89x0 EEPROM, the kernel boot
 parameter `cs89x0_media=' has been implemented.  Usage is:
 	cs89x0_media=rj45    or
 	cs89x0_media=aui     or
 	cs89x0_media=bnc
                    Linux DECnet Networking Layer Information
                   ===========================================
 1) Other documentation....
   o Project Home Pages
       http://www.chygwyn.com/DECnet/                      - Kernel info
       http://linux-decnet.sourceforge.net/                - Userland tools
       http://www.sourceforge.net/projects/linux-decnet/   - Status page
 2) Configuring the kernel
 Be sure to turn on the following options:
    CONFIG_DECNET (obviously)
    CONFIG_PROC_FS (to see what's going on)
    CONFIG_SYSCTL (for easy configuration)
 if you want to try out router support (not properly debugged yet)
 you'll need the following options as well...
    CONFIG_DECNET_ROUTER (to be able to add/delete routes)
    CONFIG_NETFILTER (will be required for the DECnet routing daemon)
    CONFIG_DECNET_ROUTE_FWMARK is optional
 Don't turn on SIOCGIFCONF support for DECnet unless you are really sure
 that you need it, in general you won't and it can cause ifconfig to
 malfunction.
 Run time configuration has changed slightly from the 2.4 system. If you
 want to configure an endnode, then the simplified procedure is as follows:
 o Set the MAC address on your ethernet card before starting _any_ other
   network protocols.
 As soon as your network card is brought into the UP state, DECnet should
 start working. If you need something more complicated or are unsure how
 to set the MAC address, see the next section. Also all configurations which
 worked with 2.4 will work under 2.5 with no change.
 3) Command line options
 You can set a DECnet address on the kernel command line for compatibility
 with the 2.4 configuration procedure, but in general it's not needed any more.
 If you do st a DECnet address on the command line, it has only one purpose
 which is that its added to the addresses on the loopback device.
 With 2.4 kernels, DECnet would only recognise addresses as local if they
 were added to the loopback device. In 2.5, any local interface address
 can be used to loop back to the local machine. Of course this does not
 prevent you adding further addresses to the loopback device if you
 want to.
 N.B. Since the address list of an interface determines the addresses for
 which "hello" messages are sent, if you don't set an address on the loopback
 interface then you won't see any entries in /proc/net/neigh for the local
 host until such time as you start a connection. This doesn't affect the
 operation of the local communications in any other way though.
 The kernel command line takes options looking like the following:
    decnet=1,2
 the two numbers are the node address 1,2 = 1.2 For 2.2.xx kernels
 and early 2.3.xx kernels, you must use a comma when specifying the
 DECnet address like this. For more recent 2.3.xx kernels, you may
 use almost any character except space, although a `.` would be the most
 obvious choice :-)
 There used to be a third number specifying the node type. This option
 has gone away in favour of a per interface node type. This is now set
 using /proc/sys/net/decnet/conf/<dev>/forwarding. This file can be
 set with a single digit, 0=EndNode, 1=L1 Router and  2=L2 Router.
 There are also equivalent options for modules. The node address can
 also be set through the /proc/sys/net/decnet/ files, as can other system
 parameters.
 Currently the only supported devices are ethernet and ip_gre. The
 ethernet address of your ethernet card has to be set according to the DECnet
 address of the node in order for it to be autoconfigured (and then appear in
 /proc/net/decnet_dev). There is a utility available at the above
 FTP sites called dn2ethaddr which can compute the correct ethernet
-address to use. The address can be set by ifconfig either before at
+address to use. The address can be set by ifconfig either before or
 at the time the device is brought up. If you are using RedHat you can
 add the line:
    MACADDR=AA:00:04:00:03:04
 or something similar, to /etc/sysconfig/network-scripts/ifcfg-eth0 or
 wherever your network card's configuration lives. Setting the MAC address
 of your ethernet card to an address starting with "hi-ord" will cause a
 DECnet address which matches to be added to the interface (which you can
 verify with iproute2).
 The default device for routing can be set through the /proc filesystem
 by setting /proc/sys/net/decnet/default_device to the
 device you want DECnet to route packets out of when no specific route
 is available. Usually this will be eth0, for example:
    echo -n "eth0" >/proc/sys/net/decnet/default_device
 If you don't set the default device, then it will default to the first
 ethernet card which has been autoconfigured as described above. You can
 confirm that by looking in the default_device file of course.
 There is a list of what the other files under /proc/sys/net/decnet/ do
 on the kernel patch web site (shown above).
 4) Run time kernel configuration
 This is either done through the sysctl/proc interface (see the kernel web
 pages for details on what the various options do) or through the iproute2
 package in the same way as IPv4/6 configuration is performed.
 Documentation for iproute2 is included with the package, although there is
 as yet no specific section on DECnet, most of the features apply to both
 IP and DECnet, albeit with DECnet addresses instead of IP addresses and
 a reduced functionality.
 If you want to configure a DECnet router you'll need the iproute2 package
 since its the _only_ way to add and delete routes currently. Eventually
 there will be a routing daemon to send and receive routing messages for
 each interface and update the kernel routing tables accordingly. The
 routing daemon will use netfilter to listen to routing packets, and
 rtnetlink to update the kernels routing tables. 
 The DECnet raw socket layer has been removed since it was there purely
 for use by the routing daemon which will now use netfilter (a much cleaner
 and more generic solution) instead.
 5) How can I tell if its working ?
 Here is a quick guide of what to look for in order to know if your DECnet
 kernel subsystem is working.
   - Is the node address set (see /proc/sys/net/decnet/node_address)
   - Is the node of the correct type 
                             (see /proc/sys/net/decnet/conf/<dev>/forwarding)
   - Is the Ethernet MAC address of each Ethernet card set to match
     the DECnet address. If in doubt use the dn2ethaddr utility available
     at the ftp archive.
   - If the previous two steps are satisfied, and the Ethernet card is up,
     you should find that it is listed in /proc/net/decnet_dev and also
     that it appears as a directory in /proc/sys/net/decnet/conf/. The
     loopback device (lo) should also appear and is required to communicate
     within a node.
   - If you have any DECnet routers on your network, they should appear
     in /proc/net/decnet_neigh, otherwise this file will only contain the
     entry for the node itself (if it doesn't check to see if lo is up).
   - If you want to send to any node which is not listed in the
     /proc/net/decnet_neigh file, you'll need to set the default device
     to point to an Ethernet card with connection to a router. This is
     again done with the /proc/sys/net/decnet/default_device file.
   - Try starting a simple server and client, like the dnping/dnmirror
     over the loopback interface. With luck they should communicate.
     For this step and those after, you'll need the DECnet library
     which can be obtained from the above ftp sites as well as the
     actual utilities themselves.
   - If this seems to work, then try talking to a node on your local
     network, and see if you can obtain the same results.
   - At this point you are on your own... :-)
 6) How to send a bug report
 If you've found a bug and want to report it, then there are several things
 you can do to help me work out exactly what it is that is wrong. Useful
 information (_most_ of which _is_ _essential_) includes:
 - What kernel version are you running ?
 - What version of the patch are you running ?
 - How far though the above set of tests can you get ?
 - What is in the /proc/decnet* files and /proc/sys/net/decnet/* files ?
 - Which services are you running ?
 - Which client caused the problem ?
 - How much data was being transferred ?
 - Was the network congested ?
 - How can the problem be reproduced ?
 - Can you use tcpdump to get a trace ? (N.B. Most (all?) versions of 
   tcpdump don't understand how to dump DECnet properly, so including
   the hex listing of the packet contents is _essential_, usually the -x flag.
   You may also need to increase the length grabbed with the -s flag. The
   -e flag also provides very useful information (ethernet MAC addresses))
 7) MAC FAQ
 A quick FAQ on ethernet MAC addresses to explain how Linux and DECnet
 interact and how to get the best performance from your hardware. 
 Ethernet cards are designed to normally only pass received network frames 
 to a host computer when they are addressed to it, or to the broadcast address.
 Linux has an interface which allows the setting of extra addresses for
 an ethernet card to listen to. If the ethernet card supports it, the
 filtering operation will be done in hardware, if not the extra unwanted packets
 received will be discarded by the host computer. In the latter case,
 significant processor time and bus bandwidth can be used up on a busy
 network (see the NAPI documentation for a longer explanation of these
 effects).
 DECnet makes use of this interface to allow running DECnet on an ethernet 
 card which has already been configured using TCP/IP (presumably using the 
 built in MAC address of the card, as usual) and/or to allow multiple DECnet
 addresses on each physical interface. If you do this, be aware that if your
 ethernet card doesn't support perfect hashing in its MAC address filter
 then your computer will be doing more work than required. Some cards
 will simply set themselves into promiscuous mode in order to receive
 packets from the DECnet specified addresses. So if you have one of these
 cards its better to set the MAC address of the card as described above
 to gain the best efficiency. Better still is to use a card which supports
 NAPI as well.
 8) Mailing list
 If you are keen to get involved in development, or want to ask questions
 about configuration, or even just report bugs, then there is a mailing
 list that you can join, details are at:
 http://sourceforge.net/mail/?group_id=4993
 9) Legal Info
 The Linux DECnet project team have placed their code under the GPL. The
 software is provided "as is" and without warranty express or implied.
 DECnet is a trademark of Compaq. This software is not a product of
 Compaq. We acknowledge the help of people at Compaq in providing extra
 documentation above and beyond what was previously publicly available.
 Steve Whitehouse <SteveW@ACM.org>
 Linux* Base Driver for the Intel(R) PRO/1000 Family of Adapters
 ===============================================================
 November 15, 2005
 Contents
 ========
 - In This Release
 - Identifying Your Adapter
 - Command Line Parameters
 - Speed and Duplex Configuration
 - Additional Configurations
 - Known Issues
 - Support
 In This Release
 ===============
 This file describes the Linux* Base Driver for the Intel(R) PRO/1000 Family
 of Adapters.  This driver includes support for Itanium(R)2-based systems.
 For questions related to hardware requirements, refer to the documentation
 supplied with your Intel PRO/1000 adapter. All hardware requirements listed
 apply to use with Linux.
 The following features are now available in supported kernels:
 - Native VLANs
 - Channel Bonding (teaming)
 - SNMP
 Channel Bonding documentation can be found in the Linux kernel source:
 /Documentation/networking/bonding.txt
 The driver information previously displayed in the /proc filesystem is not
 supported in this release.  Alternatively, you can use ethtool (version 1.6
 or later), lspci, and ifconfig to obtain the same information.
 Instructions on updating ethtool can be found in the section "Additional
 Configurations" later in this document.
 Identifying Your Adapter
 ========================
 For more information on how to identify your adapter, go to the Adapter &
 Driver ID Guide at:
    http://support.intel.com/support/network/adapter/pro100/21397.htm
 For the latest Intel network drivers for Linux, refer to the following
 website. In the search field, enter your adapter name or type, or use the
 networking link on the left to search for your adapter:
    http://downloadfinder.intel.com/scripts-df/support_intel.asp
 Command Line Parameters =======================
 If the driver is built as a module, the  following optional parameters
 are used by entering them on the command line with the modprobe or insmod
 command using this syntax:
     modprobe e1000 [<option>=<VAL1>,<VAL2>,...]
     insmod e1000 [<option>=<VAL1>,<VAL2>,...]
 For example, with two PRO/1000 PCI adapters, entering:
     insmod e1000 TxDescriptors=80,128
 loads the e1000 driver with 80 TX descriptors for the first adapter and 128
 TX descriptors for the second adapter.
 The default value for each parameter is generally the recommended setting,
 unless otherwise noted.
 NOTES:  For more information about the AutoNeg, Duplex, and Speed
        parameters, see the "Speed and Duplex Configuration" section in
        this document.
        For more information about the InterruptThrottleRate,
        RxIntDelay, TxIntDelay, RxAbsIntDelay, and TxAbsIntDelay
        parameters, see the application note at:
        http://www.intel.com/design/network/applnots/ap450.htm
        A descriptor describes a data buffer and attributes related to
        the data buffer. This information is accessed by the hardware.
 AutoNeg
 -------
 (Supported only on adapters with copper connections)
 Valid Range:   0x01-0x0F, 0x20-0x2F
 Default Value: 0x2F
 This parameter is a bit mask that specifies which speed and duplex
 settings the board advertises. When this parameter is used, the Speed
 and Duplex parameters must not be specified.
 NOTE:  Refer to the Speed and Duplex section of this readme for more
       information on the AutoNeg parameter.
 Duplex
 ------
 (Supported only on adapters with copper connections)
 Valid Range:   0-2 (0=auto-negotiate, 1=half, 2=full)
 Default Value: 0
 Defines the direction in which data is allowed to flow. Can be either
 one or two-directional. If both Duplex and the link partner are set to
 auto-negotiate, the board auto-detects the correct duplex. If the link
 partner is forced (either full or half), Duplex defaults to half-duplex.
 FlowControl
 ----------
 Valid Range:   0-3 (0=none, 1=Rx only, 2=Tx only, 3=Rx&Tx)
 Default Value: Reads flow control settings from the EEPROM
 This parameter controls the automatic generation(Tx) and response(Rx)
 to Ethernet PAUSE frames.
 InterruptThrottleRate
 ---------------------
 (not supported on Intel 82542, 82543 or 82544-based adapters)
 Valid Range:   100-100000 (0=off, 1=dynamic)
 Default Value: 8000
 This value represents the maximum number of interrupts per second the
 controller generates. InterruptThrottleRate is another setting used in
 interrupt moderation. Dynamic mode uses a heuristic algorithm to adjust
 InterruptThrottleRate based on the current traffic load.
 NOTE:  InterruptThrottleRate takes precedence over the TxAbsIntDelay and
       RxAbsIntDelay parameters. In other words, minimizing the receive
       and/or transmit absolute delays does not force the controller to
       generate more interrupts than what the Interrupt Throttle Rate
       allows.
 CAUTION:  If you are using the Intel PRO/1000 CT Network Connection
          (controller 82547), setting InterruptThrottleRate to a value
          greater than 75,000, may hang (stop transmitting) adapters
          under certain network conditions. If this occurs a NETDEV
          WATCHDOG message is logged in the system event log. In
          addition, the controller is automatically reset, restoring
          the network connection. To eliminate the potential for the
          hang, ensure that InterruptThrottleRate is set no greater
          than 75,000 and is not set to 0.
 NOTE:  When e1000 is loaded with default settings and multiple adapters
       are in use simultaneously, the CPU utilization may increase non-
       linearly. In order to limit the CPU utilization without impacting
       the overall throughput, we recommend that you load the driver as
       follows:
           insmod e1000.o InterruptThrottleRate=3000,3000,3000
       This sets the InterruptThrottleRate to 3000 interrupts/sec for
       the first, second, and third instances of the driver. The range
       of 2000 to 3000 interrupts per second works on a majority of
       systems and is a good starting point, but the optimal value will
       be platform-specific. If CPU utilization is not a concern, use
       RX_POLLING (NAPI) and default driver settings.
 RxDescriptors
 -------------
 Valid Range:   80-256 for 82542 and 82543-based adapters
               80-4096 for all other supported adapters
 Default Value: 256
 This value specifies the number of receive descriptors allocated by the
 driver. Increasing this value allows the driver to buffer more incoming
 packets.  Each descriptor is 16 bytes.  A receive buffer is also
 allocated for each descriptor and is 2048.
 RxIntDelay
 ----------
 Valid Range:   0-65535 (0=off)
 Default Value: 0
 This value delays the generation of receive interrupts in units of 1.024
 microseconds.  Receive interrupt reduction can improve CPU efficiency if
 properly tuned for specific network traffic. Increasing this value adds
 extra latency to frame reception and can end up decreasing the throughput
 of TCP traffic. If the system is reporting dropped receives, this value
 may be set too high, causing the driver to run out of available receive
 descriptors.
 CAUTION:  When setting RxIntDelay to a value other than 0, adapters may
          hang (stop transmitting) under certain network conditions. If
          this occurs a NETDEV WATCHDOG message is logged in the system
          event log. In addition, the controller is automatically reset,
          restoring the network connection. To eliminate the potential
          for the hang ensure that RxIntDelay is set to 0.
 RxAbsIntDelay
 -------------
 (This parameter is supported only on 82540, 82545 and later adapters.)
 Valid Range:   0-65535 (0=off)
 Default Value: 128
 This value, in units of 1.024 microseconds, limits the delay in which a
 receive interrupt is generated. Useful only if RxIntDelay is non-zero,
 this value ensures that an interrupt is generated after the initial
 packet is received within the set amount of time.  Proper tuning,
 along with RxIntDelay, may improve traffic throughput in specific network
 conditions.
 Speed
 -----
 (This parameter is supported only on adapters with copper connections.)
 Valid Settings: 0, 10, 100, 1000
 Default Value:  0 (auto-negotiate at all supported speeds)
 Speed forces the line speed to the specified value in megabits per second
 (Mbps). If this parameter is not specified or is set to 0 and the link
 partner is set to auto-negotiate, the board will auto-detect the correct
 speed. Duplex should also be set when Speed is set to either 10 or 100.
 TxDescriptors
 -------------
 Valid Range:   80-256 for 82542 and 82543-based adapters
               80-4096 for all other supported adapters
 Default Value: 256
 This value is the number of transmit descriptors allocated by the driver.
 Increasing this value allows the driver to queue more transmits. Each
 descriptor is 16 bytes.
 NOTE:  Depending on the available system resources, the request for a
       higher number of transmit descriptors may be denied.  In this case,
       use a lower number.
 TxIntDelay
 ----------
 Valid Range:   0-65535 (0=off)
 Default Value: 64
 This value delays the generation of transmit interrupts in units of
 1.024 microseconds. Transmit interrupt reduction can improve CPU
 efficiency if properly tuned for specific network traffic. If the
 system is reporting dropped transmits, this value may be set too high
 causing the driver to run out of available transmit descriptors.
 TxAbsIntDelay
 -------------
 (This parameter is supported only on 82540, 82545 and later adapters.)
 Valid Range:   0-65535 (0=off)
 Default Value: 64
 This value, in units of 1.024 microseconds, limits the delay in which a
 transmit interrupt is generated. Useful only if TxIntDelay is non-zero,
 this value ensures that an interrupt is generated after the initial
 packet is sent on the wire within the set amount of time.  Proper tuning,
 along with TxIntDelay, may improve traffic throughput in specific
 network conditions.
 XsumRX
 ------
 (This parameter is NOT supported on the 82542-based adapter.)
 Valid Range:   0-1
 Default Value: 1
 A value of '1' indicates that the driver should enable IP checksum
 offload for received packets (both UDP and TCP) to the adapter hardware.
 Speed and Duplex Configuration
 ==============================
 Three keywords are used to control the speed and duplex configuration.
 These keywords are Speed, Duplex, and AutoNeg.
 If the board uses a fiber interface, these keywords are ignored, and the
 fiber interface board only links at 1000 Mbps full-duplex.
 For copper-based boards, the keywords interact as follows:
  The default operation is auto-negotiate. The board advertises all
  supported speed and duplex combinations, and it links at the highest
  common speed and duplex mode IF the link partner is set to auto-negotiate.
  If Speed = 1000, limited auto-negotiation is enabled and only 1000 Mbps
  is advertised (The 1000BaseT spec requires auto-negotiation.)
  If Speed = 10 or 100, then both Speed and Duplex should be set. Auto-
  negotiation is disabled, and the AutoNeg parameter is ignored. Partner
  SHOULD also be forced.
 The AutoNeg parameter is used when more control is required over the
 auto-negotiation process.  It should be used when you wish to control which
 speed and duplex combinations are advertised during the auto-negotiation
 process.
 The parameter may be specified as either a decimal or hexidecimal value as
 determined by the bitmap below.
 Bit position   7      6      5       4       3      2      1       0
 Decimal Value  128    64     32      16      8      4      2       1
 Hex value      80     40     20      10      8      4      2       1
 Speed (Mbps)   N/A    N/A    1000    N/A     100    100    10      10
 Duplex                       Full            Full   Half   Full    Half
 Some examples of using AutoNeg:
  modprobe e1000 AutoNeg=0x01 (Restricts autonegotiation to 10 Half)
  modprobe e1000 AutoNeg=1 (Same as above)
  modprobe e1000 AutoNeg=0x02 (Restricts autonegotiation to 10 Full)
  modprobe e1000 AutoNeg=0x03 (Restricts autonegotiation to 10 Half or 10 Full)
  modprobe e1000 AutoNeg=0x04 (Restricts autonegotiation to 100 Half)
  modprobe e1000 AutoNeg=0x05 (Restricts autonegotiation to 10 Half or 100
  Half)
  modprobe e1000 AutoNeg=0x020 (Restricts autonegotiation to 1000 Full)
  modprobe e1000 AutoNeg=32 (Same as above)
 Note that when this parameter is used, Speed and Duplex must not be specified.
 If the link partner is forced to a specific speed and duplex, then this
 parameter should not be used.  Instead, use the Speed and Duplex parameters
 previously mentioned to force the adapter to the same speed and duplex.
 Additional Configurations
 =========================
  Configuring the Driver on Different Distributions
  -------------------------------------------------
  Configuring a network driver to load properly when the system is started
  is distribution dependent. Typically, the configuration process involves
  adding an alias line to /etc/modules.conf or /etc/modprobe.conf as well
  as editing other system startup scripts and/or configuration files. Many
  popular Linux distributions ship with tools to make these changes for you.
  To learn the proper way to configure a network device for your system,
  refer to your distribution documentation. If during this process you are
  asked for the driver or module name, the name for the Linux Base Driver
  for the Intel PRO/1000 Family of Adapters is e1000.
  As an example, if you install the e1000 driver for two PRO/1000 adapters
  (eth0 and eth1) and set the speed and duplex to 10full and 100half, add
-  the following to modules.conf or or modprobe.conf:
+  the following to modules.conf or modprobe.conf:
       alias eth0 e1000
       alias eth1 e1000
       options e1000 Speed=10,100 Duplex=2,1
  Viewing Link Messages
  ---------------------
  Link messages will not be displayed to the console if the distribution is
  restricting system messages. In order to see network driver link messages
  on your console, set dmesg to eight by entering the following:
       dmesg -n 8
  NOTE: This setting is not saved across reboots.
  Jumbo Frames
  ------------
  The driver supports Jumbo Frames for all adapters except 82542 and
  82573-based adapters. Jumbo Frames support is enabled by changing the
  MTU to a value larger than the default of 1500. Use the ifconfig command
  to increase the MTU size. For example:
       ifconfig eth<x> mtu 9000 up
  This setting is not saved across reboots.  It can be made permanent if
  you add:
       MTU=9000
   to the file /etc/sysconfig/network-scripts/ifcfg-eth<x>.  This example
   applies to the Red Hat distributions; other distributions may store this
   setting in a different location.
  Notes:
  - To enable Jumbo Frames, increase the MTU size on the interface beyond
    1500.
  - The maximum MTU setting for Jumbo Frames is 16110. This value coincides
    with the maximum Jumbo Frames size of 16128.
  - Using Jumbo Frames at 10 or 100 Mbps may result in poor performance or
    loss of link.
  - Some Intel gigabit adapters that support Jumbo Frames have a frame size
    limit of 9238 bytes, with a corresponding MTU size limit of 9216 bytes.
    The adapters with this limitation are based on the Intel 82571EB and
    82572EI controllers, which correspond to these product names:
     Intel® PRO/1000 PT Dual Port Server Adapter
     Intel® PRO/1000 PF Dual Port Server Adapter
     Intel® PRO/1000 PT Server Adapter
     Intel® PRO/1000 PT Desktop Adapter
     Intel® PRO/1000 PF Server Adapter
  - The Intel PRO/1000 PM Network Connection does not support jumbo frames.
  Ethtool
  -------
  The driver utilizes the ethtool interface for driver configuration and
  diagnostics, as well as displaying statistical information.  Ethtool
  version 1.6 or later is required for this functionality.
  The latest release of ethtool can be found from
  http://sourceforge.net/projects/gkernel.
  NOTE: Ethtool 1.6 only supports a limited set of ethtool options. Support
  for a more complete ethtool feature set can be enabled by upgrading
  ethtool to ethtool-1.8.1.
  Enabling Wake on LAN* (WoL)
  ---------------------------
  WoL is configured through the Ethtool* utility. Ethtool is included with
  all versions of Red Hat after Red Hat 7.2. For other Linux distributions,
  download and install Ethtool from the following website:
  http://sourceforge.net/projects/gkernel.
  For instructions on enabling WoL with Ethtool, refer to the website listed
  above.
  WoL will be enabled on the system during the next shut down or reboot.
  For this driver version, in order to enable WoL, the e1000 driver must be
  loaded when shutting down or rebooting the system.
  NAPI
  ----
  NAPI (Rx polling mode) is supported in the e1000 driver. NAPI is enabled
  or disabled based on the configuration of the kernel. To override
  the default, use the following compile-time flags.
  To enable NAPI, compile the driver module, passing in a configuration option:
       make CFLAGS_EXTRA=-DE1000_NAPI install
  To disable NAPI, compile the driver module, passing in a configuration option:
       make CFLAGS_EXTRA=-DE1000_NO_NAPI install
  See www.cyberus.ca/~hadi/usenix-paper.tgz for more information on NAPI.
 Known Issues
 ============
  Jumbo Frames System Requirement
  -------------------------------
  Memory allocation failures have been observed on Linux systems with 64 MB
  of RAM or less that are running Jumbo Frames. If you are using Jumbo
  Frames, your system may require more than the advertised minimum
  requirement of 64 MB of system memory.
  Performance Degradation with Jumbo Frames
  -----------------------------------------
  Degradation in throughput performance may be observed in some Jumbo frames
  environments. If this is observed, increasing the application's socket
  buffer size and/or increasing the /proc/sys/net/ipv4/tcp_*mem entry values
  may help. See the specific application manual and
  /usr/src/linux*/Documentation/
  networking/ip-sysctl.txt for more details.
  Jumbo frames on Foundry BigIron 8000 switch
  -------------------------------------------
  There is a known issue using Jumbo frames when connected to a Foundry
  BigIron 8000 switch. This is a 3rd party limitation. If you experience
  loss of packets, lower the MTU size.
  Multiple Interfaces on Same Ethernet Broadcast Network
  ------------------------------------------------------
  Due to the default ARP behavior on Linux, it is not possible to have
  one system on two IP networks in the same Ethernet broadcast domain
  (non-partitioned switch) behave as expected. All Ethernet interfaces
  will respond to IP traffic for any IP address assigned to the system.
  This results in unbalanced receive traffic.
  If you have multiple interfaces in a server, either turn on ARP
  filtering by entering:
      echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
  (this only works if your kernel's version is higher than 2.4.5),
  NOTE: This setting is not saved across reboots. The configuration
  change can be made permanent by adding the line:
      net.ipv4.conf.all.arp_filter = 1
  to the file /etc/sysctl.conf
        or,
  install the interfaces in separate broadcast domains (either in
  different switches or in a switch partitioned to VLANs).
  82541/82547 can't link or are slow to link with some link partners
  -----------------------------------------------------------------
  There is a known compatibility issue with 82541/82547 and some
  low-end switches where the link will not be established, or will
  be slow to establish.  In particular, these switches are known to
  be incompatible with 82541/82547:
      Planex FXG-08TE
      I-O Data ETG-SH8
  To workaround this issue, the driver can be compiled with an override
  of the PHY's master/slave setting.  Forcing master or forcing slave
  mode will improve time-to-link.
      # make EXTRA_CFLAGS=-DE1000_MASTER_SLAVE=<n>
  Where <n> is:
      0 = Hardware default
      1 = Master mode
      2 = Slave mode
      3 = Auto master/slave
  Disable rx flow control with ethtool
  ------------------------------------
  In order to disable receive flow control using ethtool, you must turn
  off auto-negotiation on the same command line.
  For example:
     ethtool -A eth? autoneg off rx off
 Support
 =======
 For general information, go to the Intel support website at:
    http://support.intel.com
    or the Intel Wired Networking project hosted by Sourceforge at:
    http://sourceforge.net/projects/e1000
 If an issue is identified with the released source code on the supported
 kernel with a supported adapter, email the specific information related
 to the issue to e1000-devel@lists.sourceforge.net
 License
 =======
 This software program is released under the terms of a license agreement
 between you ('Licensee') and Intel. Do not use or load this software or any
 associated materials (collectively, the 'Software') until you have carefully
 read the full terms and conditions of the file COPYING located in this software
 package. By loading or using the Software, you agree to the terms of this
 Agreement. If you do not agree with the terms of this Agreement, do not
 install or use the Software.
 * Other names and brands may be claimed as the property of others.
 Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.
 Contents
 =======
 - 1.  Introduction
 - 2.  Identifying the adapter/interface
 - 3.  Features supported
 - 4.  Command line parameters
 - 5.  Performance suggestions
 - 6.  Available Downloads 
 1.	Introduction:
 This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
 Xframe II PCI-X 2.0 adapters. It supports several features 
 such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
 See below for complete list of features.
 All features are supported for both IPv4 and IPv6.
 2.	Identifying the adapter/interface:
 a. Insert the adapter(s) in your system.
 b. Build and load driver 
 # insmod s2io.ko
 c. View log messages
 # dmesg | tail -40
 You will see messages similar to:
 eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
 eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
 eth4: Device is on 64 bit 133MHz PCIX(M1) bus
 The above messages identify the adapter type(Xframe I/II), adapter revision,
 driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
 In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
 as well.
 To associate an interface with a physical adapter use "ethtool -p <ethX>".
 The corresponding adapter's LED will blink multiple times.
 3.	Features supported:
 a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes,
 modifiable using ifconfig command.
 b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
 and receive, TSO.
 c. Multi-buffer receive mode. Scattering of packet across multiple
 buffers. Currently driver supports 2-buffer mode which yields
 significant performance improvement on certain platforms(SGI Altix,
 IBM xSeries).
 d. MSI/MSI-X. Can be enabled on platforms which support this feature
 (IA64, Xeon) resulting in noticeable performance improvement(upto 7%
 on certain platforms).
 e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt 
 moderation.
 f. Statistics. Comprehensive MAC-level and software statistics displayed
 using "ethtool -S" option.
 g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, 
 with multiple steering options.
 4.  Command line parameters
 a. tx_fifo_num
 Number of transmit queues
 Valid range: 1-8
 Default: 1
 b. rx_ring_num
 Number of receive rings
 Valid range: 1-8
 Default: 1
 c. tx_fifo_len
 Size of each transmit queue
 Valid range: Total length of all queues should not exceed 8192
 Default: 4096
 d. rx_ring_sz 
 Size of each receive ring(in 4K blocks)
 Valid range: Limited by memory on system
 Default: 30 
 e. intr_type
 Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X)
 Valid range: 1-3
 Default: 1 
 5.  Performance suggestions
 General:
 a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
 b. Set TCP windows size to optimal value. 
 For instance, for MTU=1500 a value of 210K has been observed to result in 
 good performance.
 # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
 # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
 For MTU=9000, TCP window size of 10 MB is recommended.
 # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
 # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
 Transmit performance:
 a. By default, the driver respects BIOS settings for PCI bus parameters. 
 However, you may want to experiment with PCI bus parameters 
 max-split-transactions(MOST) and MMRBC (use setpci command). 
 A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.  
 It could be different for your hardware.  
 Set MMRBC to 4K**.
 For example you can set 
 For opteron
 #setpci -d 17d5:* 62=1d 
 For Itanium
 #setpci -d 17d5:* 62=3d 
 For detailed description of the PCI registers, please see Xframe User Guide.
 b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this 
 parameter.
 c. Turn on TSO(using "ethtool -K")
 # ethtool -K <ethX> tso on
 Receive performance:
 a. By default, the driver respects BIOS settings for PCI bus parameters. 
 However, you may want to set PCI latency timer to 248.
 #setpci -d 17d5:* LATENCY_TIMER=f8
 For detailed description of the PCI registers, please see Xframe User Guide.
 b. Use 2-buffer mode. This results in large performance boost on
-on certain platforms(eg. SGI Altix, IBM xSeries).
+certain platforms(eg. SGI Altix, IBM xSeries).
 c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to 
 set/verify this option.
 d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network 
 device support --->  Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to 
 bring down CPU utilization.
 ** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are 
 recommended as safe parameters.
 For more information, please review the AMD8131 errata at
 http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf
 6.  Available Downloads
 Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up 
 to date, also the latest "s2io" code (including support for 2.4 kernels) is 
 available via "Support" link on the Neterion site:  http://www.neterion.com.
 For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com,
 user: linuxdocs password: HALdocs
 7. Support 
 For further support please contact either your 10GbE Xframe NIC vendor (IBM, 
 HP, SGI etc.) or click on the "Support" link on the Neterion site:  
 http://www.neterion.com.
 (C)Copyright 1999-2004 Marvell(R).
 All rights reserved
 ===========================================================================
 sk98lin.txt created 13-Feb-2004
 Readme File for sk98lin v6.23
 Marvell Yukon/SysKonnect SK-98xx Gigabit Ethernet Adapter family driver for LINUX
 This file contains
 1  Overview
 2  Required Files
 3  Installation
    3.1  Driver Installation
    3.2  Inclusion of adapter at system start
 4  Driver Parameters
    4.1  Per-Port Parameters
    4.2  Adapter Parameters
 5  Large Frame Support
 6  VLAN and Link Aggregation Support (IEEE 802.1, 802.1q, 802.3ad)
 7  Troubleshooting
 ===========================================================================
 1  Overview
 ===========
 The sk98lin driver supports the Marvell Yukon and SysKonnect 
 SK-98xx/SK-95xx compliant Gigabit Ethernet Adapter on Linux. It has 
 been tested with Linux on Intel/x86 machines.
 ***
 2  Required Files
 =================
 The linux kernel source.
 No additional files required.
 ***
 3  Installation
 ===============
 It is recommended to download the latest version of the driver from the 
 SysKonnect web site www.syskonnect.com. If you have downloaded the latest
 driver, the Linux kernel has to be patched before the driver can be 
 installed. For details on how to patch a Linux kernel, refer to the 
 patch.txt file.
 3.1  Driver Installation
 ------------------------
 The following steps describe the actions that are required to install
 the driver and to start it manually. These steps should be carried
 out for the initial driver setup. Once confirmed to be ok, they can
 be included in the system start.
 NOTE 1: To perform the following tasks you need 'root' access.
 NOTE 2: In case of problems, please read the section "Troubleshooting" 
        below.
 The driver can either be integrated into the kernel or it can be compiled 
 as a module. Select the appropriate option during the kernel 
 configuration.
 Compile/use the driver as a module
 ----------------------------------
 To compile the driver, go to the directory /usr/src/linux and
 execute the command "make menuconfig" or "make xconfig" and proceed as 
 follows:
 To integrate the driver permanently into the kernel, proceed as follows:
 1. Select the menu "Network device support" and then "Ethernet(1000Mbit)"
 2. Mark "Marvell Yukon Chipset / SysKonnect SK-98xx family support" 
   with (*) 
 3. Build a new kernel when the configuration of the above options is 
   finished.
 4. Install the new kernel.
 5. Reboot your system.
 To use the driver as a module, proceed as follows:
 1. Enable 'loadable module support' in the kernel.
 2. For automatic driver start, enable the 'Kernel module loader'.
 3. Select the menu "Network device support" and then "Ethernet(1000Mbit)"
 4. Mark "Marvell Yukon Chipset / SysKonnect SK-98xx family support" 
   with (M)
 5. Execute the command "make modules".
 6. Execute the command "make modules_install".
   The appropriate modules will be installed.
 7. Reboot your system.
 Load the module manually
 ------------------------
 To load the module manually, proceed as follows:
 1. Enter "modprobe sk98lin".
 2. If a Marvell Yukon or SysKonnect SK-98xx adapter is installed in 
   your computer and you have a /proc file system, execute the command:
   "ls /proc/net/sk98lin/" 
   This should produce an output containing a line with the following 
   format:
   eth0   eth1  ...
   which indicates that your adapter has been found and initialized.
   NOTE 1: If you have more than one Marvell Yukon or SysKonnect SK-98xx 
           adapter installed, the adapters will be listed as 'eth0', 
                   'eth1', 'eth2', etc.
                   For each adapter, repeat steps 3 and 4 below.
   NOTE 2: If you have other Ethernet adapters installed, your Marvell
           Yukon or SysKonnect SK-98xx adapter will be mapped to the 
                   next available number, e.g. 'eth1'. The mapping is executed 
                   automatically.
           The module installation message (displayed either in a system
           log file or on the console) prints a line for each adapter 
           found containing the corresponding 'ethX'.
 3. Select an IP address and assign it to the respective adapter by 
   entering:
   ifconfig eth0 <ip-address>
   With this command, the adapter is connected to the Ethernet. 
   SK-98xx Gigabit Ethernet Server Adapters: The yellow LED on the adapter 
   is now active, the link status LED of the primary port is active and 
   the link status LED of the secondary port (on dual port adapters) is 
   blinking (if the ports are connected to a switch or hub).
   SK-98xx V2.0 Gigabit Ethernet Adapters: The link status LED is active.
   In addition, you will receive a status message on the console stating
   "ethX: network connection up using port Y" and showing the selected 
   connection parameters (x stands for the ethernet device number 
   (0,1,2, etc), y stands for the port name (A or B)).
   NOTE: If you are in doubt about IP addresses, ask your network
         administrator for assistance.
 4. Your adapter should now be fully operational.
   Use 'ping <otherstation>' to verify the connection to other computers 
   on your network.
 5. To check the adapter configuration view /proc/net/sk98lin/[devicename].
   For example by executing:    
   "cat /proc/net/sk98lin/eth0" 
 Unload the module
 -----------------
 To stop and unload the driver modules, proceed as follows:
 1. Execute the command "ifconfig eth0 down".
 2. Execute the command "rmmod sk98lin".
 3.2  Inclusion of adapter at system start
 -----------------------------------------
 Since a large number of different Linux distributions are 
 available, we are unable to describe a general installation procedure
 for the driver module.
 Because the driver is now integrated in the kernel, installation should
 be easy, using the standard mechanism of your distribution.
 Refer to the distribution's manual for installation of ethernet adapters.
 ***
 4  Driver Parameters
 ====================
 Parameters can be set at the command line after the module has been 
 loaded with the command 'modprobe'.
 In some distributions, the configuration tools are able to pass parameters
 to the driver module.
 If you use the kernel module loader, you can set driver parameters
 in the file /etc/modprobe.conf (or /etc/modules.conf in 2.4 or earlier).
 To set the driver parameters in this file, proceed as follows:
 1. Insert a line of the form :
   options sk98lin ...
   For "...", the same syntax is required as described for the command
   line parameters of modprobe below.
 2. To activate the new parameters, either reboot your computer
   or 
   unload and reload the driver.
   The syntax of the driver parameters is:
        modprobe sk98lin parameter=value1[,value2[,value3...]]
   where value1 refers to the first adapter, value2 to the second etc.
 NOTE: All parameters are case sensitive. Write them exactly as shown 
      below.
 Example:
 Suppose you have two adapters. You want to set auto-negotiation
 on the first adapter to ON and on the second adapter to OFF.
 You also want to set DuplexCapabilities on the first adapter
 to FULL, and on the second adapter to HALF.
 Then, you must enter:
        modprobe sk98lin AutoNeg_A=On,Off DupCap_A=Full,Half
 NOTE: The number of adapters that can be configured this way is
      limited in the driver (file skge.c, constant SK_MAX_CARD_PARAM).
      The current limit is 16. If you happen to install
      more adapters, adjust this and recompile.
 4.1  Per-Port Parameters
 ------------------------
 These settings are available for each port on the adapter.
 In the following description, '?' stands for the port for
 which you set the parameter (A or B).
 Speed
 -----
 Parameter:    Speed_?
 Values:       10, 100, 1000, Auto
 Default:      Auto
 This parameter is used to set the speed capabilities. It is only valid 
 for the SK-98xx V2.0 copper adapters.
 Usually, the speed is negotiated between the two ports during link 
 establishment. If this fails, a port can be forced to a specific setting
 with this parameter.
 Auto-Negotiation
 ----------------
 Parameter:    AutoNeg_?
 Values:       On, Off, Sense
 Default:      On
 The "Sense"-mode automatically detects whether the link partner supports
 auto-negotiation or not.
 Duplex Capabilities
 -------------------
 Parameter:    DupCap_?
 Values:       Half, Full, Both
 Default:      Both
 This parameters is only relevant if auto-negotiation for this port is 
 not set to "Sense". If auto-negotiation is set to "On", all three values
 are possible. If it is set to "Off", only "Full" and "Half" are allowed.
 This parameter is useful if your link partner does not support all
 possible combinations.
 Flow Control
 ------------
 Parameter:    FlowCtrl_?
 Values:       Sym, SymOrRem, LocSend, None
 Default:      SymOrRem
 This parameter can be used to set the flow control capabilities the 
 port reports during auto-negotiation. It can be set for each port 
 individually.
 Possible modes:
   -- Sym      = Symmetric: both link partners are allowed to send 
                  PAUSE frames
   -- SymOrRem = SymmetricOrRemote: both or only remote partner 
                  are allowed to send PAUSE frames
   -- LocSend  = LocalSend: only local link partner is allowed 
                  to send PAUSE frames
   -- None     = no link partner is allowed to send PAUSE frames
 NOTE: This parameter is ignored if auto-negotiation is set to "Off".
 Role in Master-Slave-Negotiation (1000Base-T only)
 --------------------------------------------------
 Parameter:    Role_?
 Values:       Auto, Master, Slave
 Default:      Auto
 This parameter is only valid for the SK-9821 and SK-9822 adapters.
 For two 1000Base-T ports to communicate, one must take the role of the
 master (providing timing information), while the other must be the 
 slave. Usually, this is negotiated between the two ports during link 
 establishment. If this fails, a port can be forced to a specific setting
 with this parameter.
 4.2  Adapter Parameters
 -----------------------
 Connection Type (SK-98xx V2.0 copper adapters only)
 ---------------
 Parameter:    ConType
 Values:       Auto, 100FD, 100HD, 10FD, 10HD
 Default:      Auto
 The parameter 'ConType' is a combination of all five per-port parameters
 within one single parameter. This simplifies the configuration of both ports
 of an adapter card! The different values of this variable reflect the most 
 meaningful combinations of port parameters.
 The following table shows the values of 'ConType' and the corresponding
 combinations of the per-port parameters:
    ConType   |  DupCap   AutoNeg   FlowCtrl   Role             Speed
    ----------+------------------------------------------------------
    Auto      |  Both     On        SymOrRem   Auto             Auto
    100FD     |  Full     Off       None       Auto (ignored)   100
    100HD     |  Half     Off       None       Auto (ignored)   100
    10FD      |  Full     Off       None       Auto (ignored)   10
    10HD      |  Half     Off       None       Auto (ignored)   10
 Stating any other port parameter together with this 'ConType' variable
 will result in a merged configuration of those settings. This due to 
 the fact, that the per-port parameters (e.g. Speed_? ) have a higher
 priority than the combined variable 'ConType'.
 NOTE: This parameter is always used on both ports of the adapter card.
 Interrupt Moderation
 --------------------
 Parameter:    Moderation
 Values:       None, Static, Dynamic
 Default:      None
 Interrupt moderation is employed to limit the maximum number of interrupts
 the driver has to serve. That is, one or more interrupts (which indicate any
 transmit or receive packet to be processed) are queued until the driver 
 processes them. When queued interrupts are to be served, is determined by the
 'IntsPerSec' parameter, which is explained later below.
 Possible modes:
   -- None - No interrupt moderation is applied on the adapter card. 
      Therefore, each transmit or receive interrupt is served immediately
      as soon as it appears on the interrupt line of the adapter card.
   -- Static - Interrupt moderation is applied on the adapter card. 
      All transmit and receive interrupts are queued until a complete
      moderation interval ends. If such a moderation interval ends, all
      queued interrupts are processed in one big bunch without any delay.
      The term 'static' reflects the fact, that interrupt moderation is
      always enabled, regardless how much network load is currently 
      passing via a particular interface. In addition, the duration of
      the moderation interval has a fixed length that never changes while
      the driver is operational.
   -- Dynamic - Interrupt moderation might be applied on the adapter card,
      depending on the load of the system. If the driver detects that the
      system load is too high, the driver tries to shield the system against 
      too much network load by enabling interrupt moderation. If - at a later
      time - the CPU utilizaton decreases again (or if the network load is 
      negligible) the interrupt moderation will automatically be disabled.
 Interrupt moderation should be used when the driver has to handle one or more
 interfaces with a high network load, which - as a consequence - leads also to a
 high CPU utilization. When moderation is applied in such high network load 
 situations, CPU load might be reduced by 20-30%.
 NOTE: The drawback of using interrupt moderation is an increase of the round-
 trip-time (RTT), due to the queueing and serving of interrupts at dedicated
 moderation times.
 Interrupts per second
 ---------------------
 Parameter:    IntsPerSec
 Values:       30...40000 (interrupts per second)
 Default:      2000
 This parameter is only used if either static or dynamic interrupt moderation
 is used on a network adapter card. Using this parameter if no moderation is
 applied will lead to no action performed.
 This parameter determines the length of any interrupt moderation interval. 
 Assuming that static interrupt moderation is to be used, an 'IntsPerSec' 
 parameter value of 2000 will lead to an interrupt moderation interval of
 500 microseconds. 
 NOTE: The duration of the moderation interval is to be chosen with care.
 At first glance, selecting a very long duration (e.g. only 100 interrupts per 
 second) seems to be meaningful, but the increase of packet-processing delay 
 is tremendous. On the other hand, selecting a very short moderation time might
 compensate the use of any moderation being applied.
 Preferred Port
 --------------
 Parameter:    PrefPort
 Values:       A, B
 Default:      A
 This is used to force the preferred port to A or B (on dual-port network 
 adapters). The preferred port is the one that is used if both are detected
 as fully functional.
 RLMT Mode (Redundant Link Management Technology)
 ------------------------------------------------
 Parameter:    RlmtMode
 Values:       CheckLinkState,CheckLocalPort, CheckSeg, DualNet
 Default:      CheckLinkState
 RLMT monitors the status of the port. If the link of the active port 
 fails, RLMT switches immediately to the standby link. The virtual link is 
 maintained as long as at least one 'physical' link is up. 
 Possible modes:
   -- CheckLinkState - Check link state only: RLMT uses the link state
      reported by the adapter hardware for each individual port to 
      determine whether a port can be used for all network traffic or 
      not.
   -- CheckLocalPort - In this mode, RLMT monitors the network path 
      between the two ports of an adapter by regularly exchanging packets
      between them. This mode requires a network configuration in which 
      the two ports are able to "see" each other (i.e. there must not be 
      any router between the ports).
   -- CheckSeg - Check local port and segmentation: This mode supports the
      same functions as the CheckLocalPort mode and additionally checks 
      network segmentation between the ports. Therefore, this mode is only
      to be used if Gigabit Ethernet switches are installed on the network
      that have been configured to use the Spanning Tree protocol. 
   -- DualNet - In this mode, ports A and B are used as separate devices. 
      If you have a dual port adapter, port A will be configured as eth0 
      and port B as eth1. Both ports can be used independently with 
      distinct IP addresses. The preferred port setting is not used. 
      RLMT is turned off.
 NOTE: RLMT modes CLP and CLPSS are designed to operate in configurations 
      where a network path between the ports on one adapter exists. 
      Moreover, they are not designed to work where adapters are connected
      back-to-back.
 ***
 5  Large Frame Support
 ======================
 The driver supports large frames (also called jumbo frames). Using large 
 frames can result in an improved throughput if transferring large amounts 
 of data.
 To enable large frames, set the MTU (maximum transfer unit) of the 
 interface to the desired value (up to 9000), execute the following 
 command:
      ifconfig eth0 mtu 9000
 This will only work if you have two adapters connected back-to-back
 or if you use a switch that supports large frames. When using a switch, 
 it should be configured to allow large frames and auto-negotiation should  
 be set to OFF. The setting must be configured on all adapters that can be 
 reached by the large frames. If one adapter is not set to receive large 
 frames, it will simply drop them.
 You can switch back to the standard ethernet frame size by executing the 
 following command:
      ifconfig eth0 mtu 1500
 To permanently configure this setting, add a script with the 'ifconfig' 
 line to the system startup sequence (named something like "S99sk98lin" 
 in /etc/rc.d/rc2.d).
 ***
 6  VLAN and Link Aggregation Support (IEEE 802.1, 802.1q, 802.3ad)
 ==================================================================
 The Marvell Yukon/SysKonnect Linux drivers are able to support VLAN and 
 Link Aggregation according to IEEE standards 802.1, 802.1q, and 802.3ad. 
 These features are only available after installation of open source 
 modules available on the Internet:
 For VLAN go to: http://www.candelatech.com/~greear/vlan.html
 For Link Aggregation go to: http://www.st.rim.or.jp/~yumo
 NOTE: SysKonnect GmbH does not offer any support for these open source 
      modules and does not take the responsibility for any kind of 
      failures or problems arising in connection with these modules.
 NOTE: Configuring Link Aggregation on a SysKonnect dual link adapter may 
      cause problems when unloading the driver.
 7  Troubleshooting
 ==================
 If any problems occur during the installation process, check the 
 following list:
 Problem:  The SK-98xx adapter cannot be found by the driver.
 Solution: In /proc/pci search for the following entry:
             'Ethernet controller: SysKonnect SK-98xx ...'
          If this entry exists, the SK-98xx or SK-98xx V2.0 adapter has 
          been found by the system and should be operational.
          If this entry does not exist or if the file '/proc/pci' is not 
          found, there may be a hardware problem or the PCI support may 
          not be enabled in your kernel.
          The adapter can be checked using the diagnostics program which 
          is available on the SysKonnect web site:
          www.syskonnect.com
          Some COMPAQ machines have problems dealing with PCI under Linux.
-          Linux. This problem is described in the 'PCI howto' document
+          This problem is described in the 'PCI howto' document
          (included in some distributions or available from the
          web, e.g. at 'www.linux.org'). 
 Problem:  Programs such as 'ifconfig' or 'route' cannot be found or the 
          error message 'Operation not permitted' is displayed.
 Reason:   You are not logged in as user 'root'.
 Solution: Logout and login as 'root' or change to 'root' via 'su'.
 Problem:  Upon use of the command 'ping <address>' the message
          "ping: sendto: Network is unreachable" is displayed.
 Reason:   Your route is not set correctly.
 Solution: If you are using RedHat, you probably forgot to set up the 
          route in the 'network configuration'.
          Check the existing routes with the 'route' command and check 
          if an entry for 'eth0' exists, and if so, if it is set correctly.
 Problem:  The driver can be started, the adapter is connected to the 
          network, but you cannot receive or transmit any packets; 
          e.g. 'ping' does not work.
 Reason:   There is an incorrect route in your routing table.
 Solution: Check the routing table with the command 'route' and read the 
          manual help pages dealing with routes (enter 'man route').
 NOTE: Although the 2.2.x kernel versions generate the routing entry 
      automatically, problems of this kind may occur here as well. We've 
      come across a situation in which the driver started correctly at 
      system start, but after the driver has been removed and reloaded,
      the route of the adapter's network pointed to the 'dummy0'device 
      and had to be corrected manually.
 Problem:  Your computer should act as a router between multiple 
          IP subnetworks (using multiple adapters), but computers in 
          other subnetworks cannot be reached.
 Reason:   Either the router's kernel is not configured for IP forwarding 
          or the routing table and gateway configuration of at least one 
          computer is not working.
 Problem:  Upon driver start, the following error message is displayed:
          "eth0: -- ERROR --
          Class: internal Software error
          Nr:    0xcc
          Msg:   SkGeInitPort() cannot init running ports"
 Reason:   You are using a driver compiled for single processor machines 
          on a multiprocessor machine with SMP (Symmetric MultiProcessor) 
          kernel.
 Solution: Configure your kernel appropriately and recompile the kernel or
          the modules.
 If your problem is not listed here, please contact SysKonnect's technical
 support for help (linux@syskonnect.de).
 When contacting our technical support, please ensure that the following 
 information is available:
 - System Manufacturer and HW Informations (CPU, Memory... )
 - PCI-Boards in your system
 - Distribution
 - Kernel version
 - Driver version
 ***
 ***End of Readme File***
                       PCI Error Recovery
                       ------------------
                        February 2, 2006
                 Current document maintainer:
             Linas Vepstas <linas@austin.ibm.com>
 Many PCI bus controllers are able to detect a variety of hardware
 PCI errors on the bus, such as parity errors on the data and address
 busses, as well as SERR and PERR errors.  Some of the more advanced
 chipsets are able to deal with these errors; these include PCI-E chipsets,
 and the PCI-host bridges found on IBM Power4 and Power5-based pSeries
 boxes. A typical action taken is to disconnect the affected device,
 halting all I/O to it.  The goal of a disconnection is to avoid system
 corruption; for example, to halt system memory corruption due to DMA's
 to "wild" addresses. Typically, a reconnection mechanism is also
 offered, so that the affected PCI device(s) are reset and put back
 into working condition. The reset phase requires coordination
 between the affected device drivers and the PCI controller chip.
 This document describes a generic API for notifying device drivers
 of a bus disconnection, and then performing error recovery.
 This API is currently implemented in the 2.6.16 and later kernels.
 Reporting and recovery is performed in several steps. First, when
 a PCI hardware error has resulted in a bus disconnect, that event
 is reported as soon as possible to all affected device drivers,
 including multiple instances of a device driver on multi-function
 cards. This allows device drivers to avoid deadlocking in spinloops,
 waiting for some i/o-space register to change, when it never will.
 It also gives the drivers a chance to defer incoming I/O as
 needed.
 Next, recovery is performed in several stages. Most of the complexity
 is forced by the need to handle multi-function devices, that is,
 devices that have multiple device drivers associated with them.
 In the first stage, each driver is allowed to indicate what type
 of reset it desires, the choices being a simple re-enabling of I/O
 or requesting a hard reset (a full electrical #RST of the PCI card).
 If any driver requests a full reset, that is what will be done.
 After a full reset and/or a re-enabling of I/O, all drivers are
 again notified, so that they may then perform any device setup/config
 that may be required.  After these have all completed, a final
 "resume normal operations" event is sent out.
 The biggest reason for choosing a kernel-based implementation rather
 than a user-space implementation was the need to deal with bus
 disconnects of PCI devices attached to storage media, and, in particular,
 disconnects from devices holding the root file system.  If the root
 file system is disconnected, a user-space mechanism would have to go
 through a large number of contortions to complete recovery. Almost all
 of the current Linux file systems are not tolerant of disconnection
 from/reconnection to their underlying block device. By contrast,
 bus errors are easy to manage in the device driver. Indeed, most
 device drivers already handle very similar recovery procedures;
 for example, the SCSI-generic layer already provides significant
 mechanisms for dealing with SCSI bus errors and SCSI bus resets.
 Detailed Design
 ---------------
 Design and implementation details below, based on a chain of
 public email discussions with Ben Herrenschmidt, circa 5 April 2005.
 The error recovery API support is exposed to the driver in the form of
 a structure of function pointers pointed to by a new field in struct
 pci_driver. A driver that fails to provide the structure is "non-aware",
 and the actual recovery steps taken are platform dependent.  The
 arch/powerpc implementation will simulate a PCI hotplug remove/add.
 This structure has the form:
 struct pci_error_handlers
 {
 	int (*error_detected)(struct pci_dev *dev, enum pci_channel_state);
 	int (*mmio_enabled)(struct pci_dev *dev);
 	int (*link_reset)(struct pci_dev *dev);
 	int (*slot_reset)(struct pci_dev *dev);
 	void (*resume)(struct pci_dev *dev);
 };
 The possible channel states are:
 enum pci_channel_state {
 	pci_channel_io_normal,  /* I/O channel is in normal state */
 	pci_channel_io_frozen,  /* I/O to channel is blocked */
 	pci_channel_io_perm_failure, /* PCI card is dead */
 };
 Possible return values are:
 enum pci_ers_result {
 	PCI_ERS_RESULT_NONE,        /* no result/none/not supported in device driver */
 	PCI_ERS_RESULT_CAN_RECOVER, /* Device driver can recover without slot reset */
 	PCI_ERS_RESULT_NEED_RESET,  /* Device driver wants slot to be reset. */
 	PCI_ERS_RESULT_DISCONNECT,  /* Device has completely failed, is unrecoverable */
 	PCI_ERS_RESULT_RECOVERED,   /* Device driver is fully recovered and operational */
 };
 A driver does not have to implement all of these callbacks; however,
 if it implements any, it must implement error_detected(). If a callback
 is not implemented, the corresponding feature is considered unsupported.
 For example, if mmio_enabled() and resume() aren't there, then it
 is assumed that the driver is not doing any direct recovery and requires
 a reset. If link_reset() is not implemented, the card is assumed as
 not care about link resets. Typically a driver will want to know about
 a slot_reset().
 The actual steps taken by a platform to recover from a PCI error
 event will be platform-dependent, but will follow the general
 sequence described below.
 STEP 0: Error Event
 -------------------
 PCI bus error is detect by the PCI hardware.  On powerpc, the slot
 is isolated, in that all I/O is blocked: all reads return 0xffffffff,
 all writes are ignored.
 STEP 1: Notification
 --------------------
 Platform calls the error_detected() callback on every instance of
 every driver affected by the error.
 At this point, the device might not be accessible anymore, depending on
 the platform (the slot will be isolated on powerpc). The driver may
 already have "noticed" the error because of a failing I/O, but this
 is the proper "synchronization point", that is, it gives the driver
 a chance to cleanup, waiting for pending stuff (timers, whatever, etc...)
 to complete; it can take semaphores, schedule, etc... everything but
 touch the device. Within this function and after it returns, the driver
 shouldn't do any new IOs. Called in task context. This is sort of a
 "quiesce" point. See note about interrupts at the end of this doc.
 All drivers participating in this system must implement this call.
 The driver must return one of the following result codes:
 		- PCI_ERS_RESULT_CAN_RECOVER:
 		  Driver returns this if it thinks it might be able to recover
 		  the HW by just banging IOs or if it wants to be given
 		  a chance to extract some diagnostic information (see
 		  mmio_enable, below).
 		- PCI_ERS_RESULT_NEED_RESET:
 		  Driver returns this if it can't recover without a hard
 		  slot reset.
 		- PCI_ERS_RESULT_DISCONNECT:
 		  Driver returns this if it doesn't want to recover at all.
 The next step taken will depend on the result codes returned by the
 drivers.
 If all drivers on the segment/slot return PCI_ERS_RESULT_CAN_RECOVER,
 then the platform should re-enable IOs on the slot (or do nothing in
 particular, if the platform doesn't isolate slots), and recovery
 proceeds to STEP 2 (MMIO Enable).
 If any driver requested a slot reset (by returning PCI_ERS_RESULT_NEED_RESET),
 then recovery proceeds to STEP 4 (Slot Reset).
 If the platform is unable to recover the slot, the next step
 is STEP 6 (Permanent Failure).
 >>> The current powerpc implementation assumes that a device driver will
 >>> *not* schedule or semaphore in this routine; the current powerpc
 >>> implementation uses one kernel thread to notify all devices;
 >>> thus, if one device sleeps/schedules, all devices are affected.
 >>> Doing better requires complex multi-threaded logic in the error
 >>> recovery implementation (e.g. waiting for all notification threads
 >>> to "join" before proceeding with recovery.)  This seems excessively
 >>> complex and not worth implementing.
 >>> The current powerpc implementation doesn't much care if the device
 >>> attempts I/O at this point, or not.  I/O's will fail, returning
 >>> a value of 0xff on read, and writes will be dropped. If the device
 >>> driver attempts more than 10K I/O's to a frozen adapter, it will
 >>> assume that the device driver has gone into an infinite loop, and
->>> it will panic the the kernel. There doesn't seem to be any other
+>>> it will panic the kernel. There doesn't seem to be any other
 >>> way of stopping a device driver that insists on spinning on I/O.
 STEP 2: MMIO Enabled
 -------------------
 The platform re-enables MMIO to the device (but typically not the
 DMA), and then calls the mmio_enabled() callback on all affected
 device drivers.
 This is the "early recovery" call. IOs are allowed again, but DMA is
 not (hrm... to be discussed, I prefer not), with some restrictions. This
 is NOT a callback for the driver to start operations again, only to
 peek/poke at the device, extract diagnostic information, if any, and
 eventually do things like trigger a device local reset or some such,
 but not restart operations. This is callback is made if all drivers on
 a segment agree that they can try to recover and if no automatic link reset
 was performed by the HW. If the platform can't just re-enable IOs without
 a slot reset or a link reset, it wont call this callback, and instead
 will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
 >>> The following is proposed; no platform implements this yet:
 >>> Proposal: All I/O's should be done _synchronously_ from within
 >>> this callback, errors triggered by them will be returned via
 >>> the normal pci_check_whatever() API, no new error_detected()
 >>> callback will be issued due to an error happening here. However,
 >>> such an error might cause IOs to be re-blocked for the whole
 >>> segment, and thus invalidate the recovery that other devices
 >>> on the same segment might have done, forcing the whole segment
 >>> into one of the next states, that is, link reset or slot reset.
 The driver should return one of the following result codes:
 		- PCI_ERS_RESULT_RECOVERED
 		  Driver returns this if it thinks the device is fully
 		  functional and thinks it is ready to start
 		  normal driver operations again. There is no
 		  guarantee that the driver will actually be
 		  allowed to proceed, as another driver on the
 		  same segment might have failed and thus triggered a
 		  slot reset on platforms that support it.
 		- PCI_ERS_RESULT_NEED_RESET
 		  Driver returns this if it thinks the device is not
 		  recoverable in it's current state and it needs a slot
 		  reset to proceed.
 		- PCI_ERS_RESULT_DISCONNECT
 		  Same as above. Total failure, no recovery even after
 		  reset driver dead. (To be defined more precisely)
 The next step taken depends on the results returned by the drivers.
 If all drivers returned PCI_ERS_RESULT_RECOVERED, then the platform
 proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations).
 If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform
 proceeds to STEP 4 (Slot Reset)
 >>> The current powerpc implementation does not implement this callback.
 STEP 3: Link Reset
 ------------------
 The platform resets the link, and then calls the link_reset() callback
 on all affected device drivers.  This is a PCI-Express specific state
 and is done whenever a non-fatal error has been detected that can be
 "solved" by resetting the link. This call informs the driver of the
 reset and the driver should check to see if the device appears to be
 in working condition.
 The driver is not supposed to restart normal driver I/O operations
 at this point.  It should limit itself to "probing" the device to
 check it's recoverability status. If all is right, then the platform
 will call resume() once all drivers have ack'd link_reset().
 	Result codes:
 		(identical to STEP 3 (MMIO Enabled)
 The platform then proceeds to either STEP 4 (Slot Reset) or STEP 5
 (Resume Operations).
 >>> The current powerpc implementation does not implement this callback.
 STEP 4: Slot Reset
 ------------------
 The platform performs a soft or hard reset of the device, and then
 calls the slot_reset() callback.
 A soft reset consists of asserting the adapter #RST line and then
 restoring the PCI BAR's and PCI configuration header to a state
 that is equivalent to what it would be after a fresh system
 power-on followed by power-on BIOS/system firmware initialization.
 If the platform supports PCI hotplug, then the reset might be
 performed by toggling the slot electrical power off/on.
 It is important for the platform to restore the PCI config space
 to the "fresh poweron" state, rather than the "last state". After
 a slot reset, the device driver will almost always use its standard
 device initialization routines, and an unusual config space setup
 may result in hung devices, kernel panics, or silent data corruption.
 This call gives drivers the chance to re-initialize the hardware
 (re-download firmware, etc.).  At this point, the driver may assume
 that he card is in a fresh state and is fully functional. In
 particular, interrupt generation should work normally.
 Drivers should not yet restart normal I/O processing operations
 at this point.  If all device drivers report success on this
 callback, the platform will call resume() to complete the sequence,
 and let the driver restart normal I/O processing.
 A driver can still return a critical failure for this function if
 it can't get the device operational after reset.  If the platform
 previously tried a soft reset, it might now try a hard reset (power
 cycle) and then call slot_reset() again.  It the device still can't
 be recovered, there is nothing more that can be done;  the platform
 will typically report a "permanent failure" in such a case.  The
 device will be considered "dead" in this case.
 Drivers for multi-function cards will need to coordinate among
 themselves as to which driver instance will perform any "one-shot"
 or global device initialization. For example, the Symbios sym53cxx2
 driver performs device init only from PCI function 0:
 +       if (PCI_FUNC(pdev->devfn) == 0)
 +               sym_reset_scsi_bus(np, 0);
 	Result codes:
 		- PCI_ERS_RESULT_DISCONNECT
 		Same as above.
 Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent
 Failure).
 >>> The current powerpc implementation does not currently try a
 >>> power-cycle reset if the driver returned PCI_ERS_RESULT_DISCONNECT.
 >>> However, it probably should.
 STEP 5: Resume Operations
 -------------------------
 The platform will call the resume() callback on all affected device
 drivers if all drivers on the segment have returned
 PCI_ERS_RESULT_RECOVERED from one of the 3 previous callbacks.
 The goal of this callback is to tell the driver to restart activity,
 that everything is back and running. This callback does not return
 a result code.
 At this point, if a new error happens, the platform will restart
 a new error recovery sequence.
 STEP 6: Permanent Failure
 -------------------------
 A "permanent failure" has occurred, and the platform cannot recover
 the device.  The platform will call error_detected() with a
 pci_channel_state value of pci_channel_io_perm_failure.
 The device driver should, at this point, assume the worst. It should
 cancel all pending I/O, refuse all new I/O, returning -EIO to
 higher layers. The device driver should then clean up all of its
 memory and remove itself from kernel operations, much as it would
 during system shutdown.
 The platform will typically notify the system operator of the
 permanent failure in some way.  If the device is hotplug-capable,
 the operator will probably want to remove and replace the device.
 Note, however, not all failures are truly "permanent". Some are
 caused by over-heating, some by a poorly seated card. Many
 PCI error events are caused by software bugs, e.g. DMA's to
 wild addresses or bogus split transactions due to programming
 errors. See the discussion in powerpc/eeh-pci-error-recovery.txt
 for additional detail on real-life experience of the causes of
 software errors.
 Conclusion; General Remarks
 ---------------------------
 The way those callbacks are called is platform policy. A platform with
 no slot reset capability may want to just "ignore" drivers that can't
 recover (disconnect them) and try to let other cards on the same segment
 recover. Keep in mind that in most real life cases, though, there will
 be only one driver per segment.
 Now, a note about interrupts. If you get an interrupt and your
 device is dead or has been isolated, there is a problem :)
 The current policy is to turn this into a platform policy.
 That is, the recovery API only requires that:
 - There is no guarantee that interrupt delivery can proceed from any
 device on the segment starting from the error detection and until the
 resume callback is sent, at which point interrupts are expected to be
 fully operational.
 - There is no guarantee that interrupt delivery is stopped, that is,
 a driver that gets an interrupt after detecting an error, or that detects
 an error within the interrupt handler such that it prevents proper
 ack'ing of the interrupt (and thus removal of the source) should just
 return IRQ_NOTHANDLED. It's up to the platform to deal with that
 condition, typically by masking the IRQ source during the duration of
 the error handling. It is expected that the platform "knows" which
 interrupts are routed to error-management capable slots and can deal
 with temporarily disabling that IRQ number during error processing (this
 isn't terribly complex). That means some IRQ latency for other devices
 sharing the interrupt, but there is simply no other way. High end
 platforms aren't supposed to share interrupts between many devices
 anyway :)
 >>> Implementation details for the powerpc platform are discussed in
 >>> the file Documentation/powerpc/eeh-pci-error-recovery.txt
 >>> As of this writing, there are six device drivers with patches
 >>> implementing error recovery. Not all of these patches are in
 >>> mainline yet. These may be used as "examples":
 >>>
 >>> drivers/scsi/ipr.c
 >>> drivers/scsi/sym53cxx_2
 >>> drivers/next/e100.c
 >>> drivers/net/e1000
 >>> drivers/net/ixgb
 >>> drivers/net/s2io.c
 The End
 -------
 Some warnings, first.
 * BIG FAT WARNING *********************************************************
 *
 * If you touch anything on disk between suspend and resume...
 *				...kiss your data goodbye.
 *
 * If you do resume from initrd after your filesystems are mounted...
 *				...bye bye root partition.
 *			[this is actually same case as above]
 *
 * If you have unsupported (*) devices using DMA, you may have some
 * problems. If your disk driver does not support suspend... (IDE does),
 * it may cause some problems, too. If you change kernel command line
 * between suspend and resume, it may do something wrong. If you change
 * your hardware while system is suspended... well, it was not good idea;
 * but it will probably only crash.
 *
 * (*) suspend/resume support is needed to make it safe.
 *
 * If you have any filesystems on USB devices mounted before software suspend,
 * they won't be accessible after resume and you may lose data, as though
 * you have unplugged the USB devices with mounted filesystems on them;
 * see the FAQ below for details.  (This is not true for more traditional
 * power states like "standby", which normally don't turn USB off.)
 You need to append resume=/dev/your_swap_partition to kernel command
 line. Then you suspend by
 echo shutdown > /sys/power/disk; echo disk > /sys/power/state
 . If you feel ACPI works pretty well on your system, you might try
 echo platform > /sys/power/disk; echo disk > /sys/power/state
 . If you have SATA disks, you'll need recent kernels with SATA suspend
 support. For suspend and resume to work, make sure your disk drivers
 are built into kernel -- not modules. [There's way to make
 suspend/resume with modular disk drivers, see FAQ, but you probably
 should not do that.]
 If you want to limit the suspend image size to N bytes, do
 echo N > /sys/power/image_size
 before suspend (it is limited to 500 MB by default).
 Article about goals and implementation of Software Suspend for Linux
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Author: G‚ábor Kuti
 Last revised: 2003-10-20 by Pavel Machek
 Idea and goals to achieve
 Nowadays it is common in several laptops that they have a suspend button. It
 saves the state of the machine to a filesystem or to a partition and switches
 to standby mode. Later resuming the machine the saved state is loaded back to
 ram and the machine can continue its work. It has two real benefits. First we
 save ourselves the time machine goes down and later boots up, energy costs
 are real high when running from batteries. The other gain is that we don't have to
 interrupt our programs so processes that are calculating something for a long
 time shouldn't need to be written interruptible.
 swsusp saves the state of the machine into active swaps and then reboots or
 powerdowns.  You must explicitly specify the swap partition to resume from with
 ``resume='' kernel option. If signature is found it loads and restores saved
 state. If the option ``noresume'' is specified as a boot parameter, it skips
 the resuming.
 In the meantime while the system is suspended you should not add/remove any
 of the hardware, write to the filesystems, etc.
 Sleep states summary
 ====================
 There are three different interfaces you can use, /proc/acpi should
 work like this:
 In a really perfect world:
 echo 1 > /proc/acpi/sleep       # for standby
 echo 2 > /proc/acpi/sleep       # for suspend to ram
 echo 3 > /proc/acpi/sleep       # for suspend to ram, but with more power conservative
 echo 4 > /proc/acpi/sleep       # for suspend to disk
 echo 5 > /proc/acpi/sleep       # for shutdown unfriendly the system
 and perhaps
 echo 4b > /proc/acpi/sleep      # for suspend to disk via s4bios
 Frequently Asked Questions
 ==========================
 Q: well, suspending a server is IMHO a really stupid thing,
 but... (Diego Zuccato):
 A: You bought new UPS for your server. How do you install it without
 bringing machine down? Suspend to disk, rearrange power cables,
 resume.
 You have your server on UPS. Power died, and UPS is indicating 30
 seconds to failure. What do you do? Suspend to disk.
 Q: Maybe I'm missing something, but why don't the regular I/O paths work?
 A: We do use the regular I/O paths. However we cannot restore the data
 to its original location as we load it. That would create an
 inconsistent kernel state which would certainly result in an oops.
 Instead, we load the image into unused memory and then atomically copy
 it back to it original location. This implies, of course, a maximum
 image size of half the amount of memory.
 There are two solutions to this:
 * require half of memory to be free during suspend. That way you can
 read "new" data onto free spots, then cli and copy
 * assume we had special "polling" ide driver that only uses memory
 between 0-640KB. That way, I'd have to make sure that 0-640KB is free
 during suspending, but otherwise it would work...
 suspend2 shares this fundamental limitation, but does not include user
 data and disk caches into "used memory" by saving them in
 advance. That means that the limitation goes away in practice.
 Q: Does linux support ACPI S4?
 A: Yes. That's what echo platform > /sys/power/disk does.
 Q: What is 'suspend2'?
 A: suspend2 is 'Software Suspend 2', a forked implementation of
 suspend-to-disk which is available as separate patches for 2.4 and 2.6
 kernels from swsusp.sourceforge.net. It includes support for SMP, 4GB
 highmem and preemption. It also has a extensible architecture that
 allows for arbitrary transformations on the image (compression,
 encryption) and arbitrary backends for writing the image (eg to swap
 or an NFS share[Work In Progress]). Questions regarding suspend2
 should be sent to the mailing list available through the suspend2
 website, and not to the Linux Kernel Mailing List. We are working
 toward merging suspend2 into the mainline kernel.
 Q: A kernel thread must voluntarily freeze itself (call 'refrigerator').
 I found some kernel threads that don't do it, and they don't freeze
 so the system can't sleep. Is this a known behavior?
 A: All such kernel threads need to be fixed, one by one. Select the
 place where the thread is safe to be frozen (no kernel semaphores
 should be held at that point and it must be safe to sleep there), and
 add:
       try_to_freeze();
 If the thread is needed for writing the image to storage, you should
 instead set the PF_NOFREEZE process flag when creating the thread (and
 be very carefull).
-Q: What is the difference between between "platform", "shutdown" and
+Q: What is the difference between "platform", "shutdown" and
 "firmware" in /sys/power/disk?
 A:
 shutdown: save state in linux, then tell bios to powerdown
 platform: save state in linux, then tell bios to powerdown and blink
          "suspended led"
 firmware: tell bios to save state itself [needs BIOS-specific suspend
 	  partition, and has very little to do with swsusp]
 "platform" is actually right thing to do, but "shutdown" is most
 reliable.
 Q: I do not understand why you have such strong objections to idea of
 selective suspend.
 A: Do selective suspend during runtime power management, that's okay. But
 it's useless for suspend-to-disk. (And I do not see how you could use
 it for suspend-to-ram, I hope you do not want that).
 Lets see, so you suggest to
 * SUSPEND all but swap device and parents
 * Snapshot
 * Write image to disk
 * SUSPEND swap device and parents
 * Powerdown
 Oh no, that does not work, if swap device or its parents uses DMA,
 you've corrupted data. You'd have to do
 * SUSPEND all but swap device and parents
 * FREEZE swap device and parents
 * Snapshot
 * UNFREEZE swap device and parents
 * Write
 * SUSPEND swap device and parents
 Which means that you still need that FREEZE state, and you get more
 complicated code. (And I have not yet introduce details like system
 devices).
 Q: There don't seem to be any generally useful behavioral
 distinctions between SUSPEND and FREEZE.
 A: Doing SUSPEND when you are asked to do FREEZE is always correct,
 but it may be unneccessarily slow. If you want your driver to stay simple,
 slowness may not matter to you. It can always be fixed later.
 For devices like disk it does matter, you do not want to spindown for
 FREEZE.
 Q: After resuming, system is paging heavily, leading to very bad interactivity.
 A: Try running
 cat `cat /proc/[0-9]*/maps | grep / | sed 's:.* /:/:' | sort -u` > /dev/null
 after resume. swapoff -a; swapon -a may also be useful.
 Q: What happens to devices during swsusp? They seem to be resumed
 during system suspend?
 A: That's correct. We need to resume them if we want to write image to
 disk. Whole sequence goes like
      Suspend part
      ~~~~~~~~~~~~
      running system, user asks for suspend-to-disk
      user processes are stopped
      suspend(PMSG_FREEZE): devices are frozen so that they don't interfere
      		      with state snapshot
      state snapshot: copy of whole used memory is taken with interrupts disabled
      resume(): devices are woken up so that we can write image to swap
      write image to swap
      suspend(PMSG_SUSPEND): suspend devices so that we can power off
      turn the power off
      Resume part
      ~~~~~~~~~~~
      (is actually pretty similar)
      running system, user asks for suspend-to-disk
      user processes are stopped (in common case there are none, but with resume-from-initrd, noone knows)
      read image from disk
      suspend(PMSG_FREEZE): devices are frozen so that they don't interfere
      		      with image restoration
      image restoration: rewrite memory with image
      resume(): devices are woken up so that system can continue
      thaw all user processes
 Q: What is this 'Encrypt suspend image' for?
 A: First of all: it is not a replacement for dm-crypt encrypted swap.
 It cannot protect your computer while it is suspended. Instead it does
 protect from leaking sensitive data after resume from suspend.
 Think of the following: you suspend while an application is running
 that keeps sensitive data in memory. The application itself prevents
 the data from being swapped out. Suspend, however, must write these
 data to swap to be able to resume later on. Without suspend encryption
 your sensitive data are then stored in plaintext on disk.  This means
 that after resume your sensitive data are accessible to all
 applications having direct access to the swap device which was used
 for suspend. If you don't need swap after resume these data can remain
 on disk virtually forever. Thus it can happen that your system gets
 broken in weeks later and sensitive data which you thought were
 encrypted and protected are retrieved and stolen from the swap device.
 To prevent this situation you should use 'Encrypt suspend image'.
 During suspend a temporary key is created and this key is used to
 encrypt the data written to disk. When, during resume, the data was
 read back into memory the temporary key is destroyed which simply
 means that all data written to disk during suspend are then
 inaccessible so they can't be stolen later on.  The only thing that
 you must then take care of is that you call 'mkswap' for the swap
 partition used for suspend as early as possible during regular
 boot. This asserts that any temporary key from an oopsed suspend or
 from a failed or aborted resume is erased from the swap device.
 As a rule of thumb use encrypted swap to protect your data while your
 system is shut down or suspended. Additionally use the encrypted
 suspend image to prevent sensitive data from being stolen after
 resume.
 Q: Why can't we suspend to a swap file?
 A: Because accessing swap file needs the filesystem mounted, and
 filesystem might do something wrong (like replaying the journal)
 during mount.
 There are few ways to get that fixed:
 1) Probably could be solved by modifying every filesystem to support
 some kind of "really read-only!" option. Patches welcome.
 2) suspend2 gets around that by storing absolute positions in on-disk
 image (and blocksize), with resume parameter pointing directly to
 suspend header.
 Q: Is there a maximum system RAM size that is supported by swsusp?
 A: It should work okay with highmem.
 Q: Does swsusp (to disk) use only one swap partition or can it use
 multiple swap partitions (aggregate them into one logical space)?
 A: Only one swap partition, sorry.
 Q: If my application(s) causes lots of memory & swap space to be used
 (over half of the total system RAM), is it correct that it is likely
 to be useless to try to suspend to disk while that app is running?
 A: No, it should work okay, as long as your app does not mlock()
 it. Just prepare big enough swap partition.
 Q: What information is useful for debugging suspend-to-disk problems?
 A: Well, last messages on the screen are always useful. If something
 is broken, it is usually some kernel driver, therefore trying with as
 little as possible modules loaded helps a lot. I also prefer people to
 suspend from console, preferably without X running. Booting with
 init=/bin/bash, then swapon and starting suspend sequence manually
 usually does the trick. Then it is good idea to try with latest
 vanilla kernel.
 Q: How can distributions ship a swsusp-supporting kernel with modular
 disk drivers (especially SATA)?
 A: Well, it can be done, load the drivers, then do echo into
 /sys/power/disk/resume file from initrd. Be sure not to mount
 anything, not even read-only mount, or you are going to lose your
 data.
 Q: How do I make suspend more verbose?
 A: If you want to see any non-error kernel messages on the virtual
 terminal the kernel switches to during suspend, you have to set the
 kernel console loglevel to at least 4 (KERN_WARNING), for example by
 doing
 	# save the old loglevel
 	read LOGLEVEL DUMMY < /proc/sys/kernel/printk
 	# set the loglevel so we see the progress bar.
 	# if the level is higher than needed, we leave it alone.
 	if [ $LOGLEVEL -lt 5 ]; then
 	        echo 5 > /proc/sys/kernel/printk
 		fi
        IMG_SZ=0
        read IMG_SZ < /sys/power/image_size
        echo -n disk > /sys/power/state
        RET=$?
        #
        # the logic here is:
        # if image_size > 0 (without kernel support, IMG_SZ will be zero),
        # then try again with image_size set to zero.
 	if [ $RET -ne 0 -a $IMG_SZ -ne 0 ]; then # try again with minimal image size
                echo 0 > /sys/power/image_size
                echo -n disk > /sys/power/state
                RET=$?
        fi
 	# restore previous loglevel
 	echo $LOGLEVEL > /proc/sys/kernel/printk
 	exit $RET
 Q: Is this true that if I have a mounted filesystem on a USB device and
 I suspend to disk, I can lose data unless the filesystem has been mounted
 with "sync"?
 A: That's right ... if you disconnect that device, you may lose data.
 In fact, even with "-o sync" you can lose data if your programs have
 information in buffers they haven't written out to a disk you disconnect,
 or if you disconnect before the device finished saving data you wrote.
 Software suspend normally powers down USB controllers, which is equivalent
 to disconnecting all USB devices attached to your system.
 Your system might well support low-power modes for its USB controllers
 while the system is asleep, maintaining the connection, using true sleep
 modes like "suspend-to-RAM" or "standby".  (Don't write "disk" to the
 /sys/power/state file; write "standby" or "mem".)  We've not seen any
 hardware that can use these modes through software suspend, although in
 theory some systems might support "platform" or "firmware" modes that
 won't break the USB connections.
 Remember that it's always a bad idea to unplug a disk drive containing a
 mounted filesystem.  That's true even when your system is asleep!  The
 safest thing is to unmount all filesystems on removable media (such USB,
 Firewire, CompactFlash, MMC, external SATA, or even IDE hotplug bays)
 before suspending; then remount them after resuming.
 Q: I upgraded the kernel from 2.6.15 to 2.6.16. Both kernels were
 compiled with the similar configuration files. Anyway I found that
 suspend to disk (and resume) is much slower on 2.6.16 compared to
 2.6.15. Any idea for why that might happen or how can I speed it up?
 A: This is because the size of the suspend image is now greater than
 for 2.6.15 (by saving more data we can get more responsive system
 after resume).
 There's the /sys/power/image_size knob that controls the size of the
 image.  If you set it to 0 (eg. by echo 0 > /sys/power/image_size as
 root), the 2.6.15 behavior should be restored.  If it is still too
 slow, take a look at suspend.sf.net -- userland suspend is faster and
 supports LZF compression to speed it up further.
 The prio_tree.c code indexes vmas using 3 different indexes:
 	* heap_index  = vm_pgoff + vm_size_in_pages : end_vm_pgoff
 	* radix_index = vm_pgoff : start_vm_pgoff
 	* size_index = vm_size_in_pages
 A regular radix-priority-search-tree indexes vmas using only heap_index and
 radix_index. The conditions for indexing are:
 	* ->heap_index >= ->left->heap_index &&
 		->heap_index >= ->right->heap_index
 	* if (->heap_index == ->left->heap_index)
 		then ->radix_index < ->left->radix_index;
 	* if (->heap_index == ->right->heap_index)
 		then ->radix_index < ->right->radix_index;
 	* nodes are hashed to left or right subtree using radix_index
 	  similar to a pure binary radix tree.
 A regular radix-priority-search-tree helps to store and query
 intervals (vmas). However, a regular radix-priority-search-tree is only
 suitable for storing vmas with different radix indices (vm_pgoff).
 Therefore, the prio_tree.c extends the regular radix-priority-search-tree
 to handle many vmas with the same vm_pgoff. Such vmas are handled in
 2 different ways: 1) All vmas with the same radix _and_ heap indices are
 linked using vm_set.list, 2) if there are many vmas with the same radix
 index, but different heap indices and if the regular radix-priority-search
 tree cannot index them all, we build an overflow-sub-tree that indexes such
 vmas using heap and size indices instead of heap and radix indices. For
 example, in the figure below some vmas with vm_pgoff = 0 (zero) are
 indexed by regular radix-priority-search-tree whereas others are pushed
 into an overflow-subtree. Note that all vmas in an overflow-sub-tree have
 the same vm_pgoff (radix_index) and if necessary we build different
 overflow-sub-trees to handle each possible radix_index. For example,
 in figure we have 3 overflow-sub-trees corresponding to radix indices
 0, 2, and 4.
 In the final tree the first few (prio_tree_root->index_bits) levels
 are indexed using heap and radix indices whereas the overflow-sub-trees below
 those levels (i.e. levels prio_tree_root->index_bits + 1 and higher) are
 indexed using heap and size indices. In overflow-sub-trees the size_index
 is used for hashing the nodes to appropriate places.
 Now, an example prio_tree:
  vmas are represented [radix_index, size_index, heap_index]
                 i.e., [start_vm_pgoff, vm_size_in_pages, end_vm_pgoff]
 level  prio_tree_root->index_bits = 3
 -----
 												_
  0			 				[0,7,7]					 |
  							/     \					 |
 				      ------------------       ------------			 |     Regular
  				     /					   \			 |  radix priority
  1		 		[1,6,7]					  [4,3,7]		 |   search tree
  				/     \					  /     \		 |
 			 -------       -----			    ------       -----		 |  heap-and-radix
 			/		    \			   /		      \		 |      indexed
  2		    [0,6,6]	 	   [2,5,7]		[5,2,7]		    [6,1,7]	 |
 		    /     \		   /     \		/     \		    /     \	 |
  3		[0,5,5]	[1,5,6]		[2,4,6]	[3,4,7]	    [4,2,6] [5,1,6]	[6,0,6]	[7,0,7]	 |
 		   /			   /		       /		   		_
                  /		          /		      /					_
  4	      [0,4,4]		      [2,3,5]		   [4,1,5]				 |
  		 /			 /		      /					 |
  5	     [0,3,3]		     [2,2,4]		  [4,0,4]				 |  Overflow-sub-trees
  		/			/							 |
  6	    [0,2,2]		    [2,1,3]							 |    heap-and-size
  	       /		       /							 |       indexed
  7	   [0,1,1]		   [2,0,2]							 |
  	      /											 |
  8	  [0,0,0]										 |
  												_
 Note that we use prio_tree_root->index_bits to optimize the height
 of the heap-and-radix indexed tree. Since prio_tree_root->index_bits is
 set according to the maximum end_vm_pgoff mapped, we are sure that all
 bits (in vm_pgoff) above prio_tree_root->index_bits are 0 (zero). Therefore,
 we only use the first prio_tree_root->index_bits as radix_index.
 Whenever index_bits is increased in prio_tree_expand, we shuffle the tree
 to make sure that the first prio_tree_root->index_bits levels of the tree
 is indexed properly using heap and radix indices.
 We do not optimize the height of overflow-sub-trees using index_bits.
 The reason is: there can be many such overflow-sub-trees and all of
 them have to be suffled whenever the index_bits increases. This may involve
 walking the whole prio_tree in prio_tree_insert->prio_tree_expand code
 path which is not desirable. Hence, we do not optimize the height of the
 heap-and-size indexed overflow-sub-trees using prio_tree->index_bits.
 Instead the overflow sub-trees are indexed using full BITS_PER_LONG bits
 of size_index. This may lead to skewed sub-trees because most of the
-higher significant bits of the size_index are likely to be be 0 (zero). In
+higher significant bits of the size_index are likely to be 0 (zero). In
 the example above, all 3 overflow-sub-trees are skewed. This may marginally
 affect the performance. However, processes rarely map many vmas with the
 same start_vm_pgoff but different end_vm_pgoffs. Therefore, we normally
 do not require overflow-sub-trees to index all vmas.
 From the above discussion it is clear that the maximum height of
 a prio_tree can be prio_tree_root->index_bits + BITS_PER_LONG.
 However, in most of the common cases we do not need overflow-sub-trees,
 so the tree height in the common cases will be prio_tree_root->index_bits.
 It is fair to mention here that the prio_tree_root->index_bits
 is increased on demand, however, the index_bits is not decreased when
 vmas are removed from the prio_tree. That's tricky to do. Hence, it's
 left as a home work problem.
 	This document gives a brief introduction to the caching
 mechanisms in the sunrpc layer that is used, in particular,
 for NFS authentication.
 CACHES
 ======
 The caching replaces the old exports table and allows for
 a wide variety of values to be caches.
 There are a number of caches that are similar in structure though
 quite possibly very different in content and use.  There is a corpus
 of common code for managing these caches.
 Examples of caches that are likely to be needed are:
  - mapping from IP address to client name
  - mapping from client name and filesystem to export options
  - mapping from UID to list of GIDs, to work around NFS's limitation
    of 16 gids.
  - mappings between local UID/GID and remote UID/GID for sites that
    do not have uniform uid assignment
  - mapping from network identify to public key for crypto authentication.
 The common code handles such things as:
   - general cache lookup with correct locking
   - supporting 'NEGATIVE' as well as positive entries
   - allowing an EXPIRED time on cache items, and removing
     items after they expire, and are no longer in-use.
   - making requests to user-space to fill in cache entries
   - allowing user-space to directly set entries in the cache
   - delaying RPC requests that depend on as-yet incomplete
     cache entries, and replaying those requests when the cache entry
     is complete.
   - clean out old entries as they expire.
 Creating a Cache
 ----------------
 1/ A cache needs a datum to store.  This is in the form of a
   structure definition that must contain a
     struct cache_head
   as an element, usually the first.
   It will also contain a key and some content.
   Each cache element is reference counted and contains
   expiry and update times for use in cache management.
 2/ A cache needs a "cache_detail" structure that
   describes the cache.  This stores the hash table, some
   parameters for cache management, and some operations detailing how
   to work with particular cache items.
   The operations requires are:
   	struct cache_head *alloc(void)
 		This simply allocates appropriate memory and returns
   		a pointer to the cache_detail embedded within the
 		structure
 	void cache_put(struct kref *)
 		This is called when the last reference to an item is
-		is dropped.  The pointer passed is to the 'ref' field
+		dropped.  The pointer passed is to the 'ref' field
 		in the cache_head.  cache_put should release any
 		references create by 'cache_init' and, if CACHE_VALID
 		is set, any references created by cache_update.
 		It should then release the memory allocated by
   		'alloc'.
        int match(struct cache_head *orig, struct cache_head *new)
 		test if the keys in the two structures match.  Return
 		1 if they do, 0 if they don't.
 	void init(struct cache_head *orig, struct cache_head *new)
 		Set the 'key' fields in 'new' from 'orig'.  This may
 		include taking references to shared objects.
 	void update(struct cache_head *orig, struct cache_head *new)
 		Set the 'content' fileds in 'new' from 'orig'.
 	int cache_show(struct seq_file *m, struct cache_detail *cd,
 			struct cache_head *h)
 		Optional.  Used to provide a /proc file that lists the
 		contents of a cache.  This should show one item,
   		usually on just one line.
 	int cache_request(struct cache_detail *cd, struct cache_head *h,
   		char **bpp, int *blen)
 		Format a request to be send to user-space for an item
   		to be instantiated.  *bpp is a buffer of size *blen.
 		bpp should be moved forward over the encoded message,
 		and  *blen should be reduced to show how much free
 		space remains.  Return 0 on success or <0 if not
 		enough room or other problem.
 	int cache_parse(struct cache_detail *cd, char *buf, int len)
 		A message from user space has arrived to fill out a
 		cache entry.  It is in 'buf' of length 'len'.
 		cache_parse should parse this, find the item in the
 		cache with sunrpc_cache_lookup, and update the item
 		with sunrpc_cache_update.
 3/ A cache needs to be registered using cache_register().  This
   includes it on a list of caches that will be regularly
   cleaned to discard old data.
 Using a cache
 -------------
 To find a value in a cache, call sunrpc_cache_lookup passing a pointer
 to the cache_head in a sample item with the 'key' fields filled in.
 This will be passed to ->match to identify the target entry.  If no
 entry is found, a new entry will be create, added to the cache, and
 marked as not containing valid data.
 The item returned is typically passed to cache_check which will check
 if the data is valid, and may initiate an up-call to get fresh data.
 cache_check will return -ENOENT in the entry is negative or if an up
 call is needed but not possible, -EAGAIN if an upcall is pending,
 or 0 if the data is valid;
 cache_check can be passed a "struct cache_req *".  This structure is
 typically embedded in the actual request and can be used to create a
 deferred copy of the request (struct cache_deferred_req).  This is
 done when the found cache item is not uptodate, but the is reason to
 believe that userspace might provide information soon.  When the cache
 item does become valid, the deferred copy of the request will be
 revisited (->revisit).  It is expected that this method will
 reschedule the request for processing.
 The value returned by sunrpc_cache_lookup can also be passed to
 sunrpc_cache_update to set the content for the item.  A second item is
 passed which should hold the content.  If the item found by _lookup
 has valid data, then it is discarded and a new item is created.  This
 saves any user of an item from worrying about content changing while
 it is being inspected.  If the item found by _lookup does not contain
 valid data, then the content is copied across and CACHE_VALID is set.
 Populating a cache
 ------------------
 Each cache has a name, and when the cache is registered, a directory
 with that name is created in /proc/net/rpc
 This directory contains a file called 'channel' which is a channel
 for communicating between kernel and user for populating the cache.
 This directory may later contain other files of interacting
 with the cache.
 The 'channel' works a bit like a datagram socket. Each 'write' is
 passed as a whole to the cache for parsing and interpretation.
 Each cache can treat the write requests differently, but it is
 expected that a message written will contain:
  - a key
  - an expiry time
  - a content.
 with the intention that an item in the cache with the give key
 should be create or updated to have the given content, and the
 expiry time should be set on that item.
 Reading from a channel is a bit more interesting.  When a cache
 lookup fails, or when it succeeds but finds an entry that may soon
 expire, a request is lodged for that cache item to be updated by
 user-space.  These requests appear in the channel file.
 Successive reads will return successive requests.
 If there are no more requests to return, read will return EOF, but a
 select or poll for read will block waiting for another request to be
 added.
 Thus a user-space helper is likely to:
  open the channel.
    select for readable
    read a request
    write a response
  loop.
 If it dies and needs to be restarted, any requests that have not been
 answered will still appear in the file and will be read by the new
 instance of the helper.
 Each cache should define a "cache_parse" method which takes a message
 written from user-space and processes it.  It should return an error
 (which propagates back to the write syscall) or 0.
 Each cache should also define a "cache_request" method which
 takes a cache item and encodes a request into the buffer
 provided.
 Note: If a cache has no active readers on the channel, and has had not
 active readers for more than 60 seconds, further requests will not be
 added to the channel but instead all lookups that do not find a valid
 entry will fail.  This is partly for backward compatibility: The
 previous nfs exports table was deemed to be authoritative and a
 failed lookup meant a definite 'no'.
 request/response format
 -----------------------
 While each cache is free to use it's own format for requests
 and responses over channel, the following is recommended as
 appropriate and support routines are available to help:
 Each request or response record should be printable ASCII
 with precisely one newline character which should be at the end.
 Fields within the record should be separated by spaces, normally one.
 If spaces, newlines, or nul characters are needed in a field they
 much be quoted.  two mechanisms are available:
 1/ If a field begins '\x' then it must contain an even number of
   hex digits, and pairs of these digits provide the bytes in the
   field.
 2/ otherwise a \ in the field must be followed by 3 octal digits
   which give the code for a byte.  Other characters are treated
   as them selves.  At the very least, space, newline, nul, and
   '\' must be quoted in this way.
                          Debugging on Linux for s/390 & z/Architecture
 			               by
 		Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com)
 		Copyright (C) 2000-2001 IBM Deutschland Entwicklung GmbH, IBM Corporation
                              Best viewed with fixed width fonts 
 Overview of Document:
 =====================
 This document is intended to give an good overview of how to debug 
 Linux for s/390 & z/Architecture. It isn't intended as a complete reference & not a
 tutorial on the fundamentals of C & assembly. It doesn't go into
 390 IO in any detail. It is intended to complement the documents in the
 reference section below & any other worthwhile references you get.
 It is intended like the Enterprise Systems Architecture/390 Reference Summary
 to be printed out & used as a quick cheat sheet self help style reference when
 problems occur.
 Contents
 ========
 Register Set
 Address Spaces on Intel Linux
 Address Spaces on Linux for s/390 & z/Architecture
 The Linux for s/390 & z/Architecture Kernel Task Structure
 Register Usage & Stackframes on Linux for s/390 & z/Architecture
 A sample program with comments
 Compiling programs for debugging on Linux for s/390 & z/Architecture
 Figuring out gcc compile errors
 Debugging Tools
 objdump
 strace
 Performance Debugging 
 Debugging under VM
 s/390 & z/Architecture IO Overview
 Debugging IO on s/390 & z/Architecture under VM
 GDB on s/390 & z/Architecture
 Stack chaining in gdb by hand
 Examining core dumps
 ldd
 Debugging modules
 The proc file system
 Starting points for debugging scripting languages etc.
 Dumptool & Lcrash
 SysRq
 References
 Special Thanks
 Register Set
 ============
 The current architectures have the following registers.
 16  General propose registers, 32 bit on s/390 64 bit on z/Architecture, r0-r15 or gpr0-gpr15 used for arithmetic & addressing. 
 16 Control registers, 32 bit on s/390 64 bit on z/Architecture, ( cr0-cr15 kernel usage only ) used for memory management,
 interrupt control,debugging control etc.
 16 Access registers ( ar0-ar15 ) 32 bit on s/390 & z/Architecture
 not used by normal programs but potentially could 
 be used as temporary storage. Their main purpose is their 1 to 1
 association with general purpose registers and are used in
 the kernel for copying data between kernel & user address spaces.
 Access register 0 ( & access register 1 on z/Architecture ( needs 64 bit 
 pointer ) ) is currently used by the pthread library as a pointer to
 the current running threads private area.
 16 64 bit floating point registers (fp0-fp15 ) IEEE & HFP floating 
 point format compliant on G5 upwards & a Floating point control reg (FPC) 
 4  64 bit registers (fp0,fp2,fp4 & fp6) HFP only on older machines.
 Note:
 Linux (currently) always uses IEEE & emulates G5 IEEE format on older machines,
 ( provided the kernel is configured for this ).
 The PSW is the most important register on the machine it
 is 64 bit on s/390 & 128 bit on z/Architecture & serves the roles of 
 a program counter (pc), condition code register,memory space designator.
 In IBM standard notation I am counting bit 0 as the MSB.
 It has several advantages over a normal program counter
 in that you can change address translation & program counter 
 in a single instruction. To change address translation,
 e.g. switching address translation off requires that you
 have a logical=physical mapping for the address you are
 currently running at.
      Bit           Value
 s/390 z/Architecture
 0       0     Reserved ( must be 0 ) otherwise specification exception occurs.
 1       1     Program Event Recording 1 PER enabled, 
 	      PER is used to facilitate debugging e.g. single stepping.
 2-4    2-4    Reserved ( must be 0 ). 
 5       5     Dynamic address translation 1=DAT on.
 6       6     Input/Output interrupt Mask
 7       7     External interrupt Mask used primarily for interprocessor signalling & 
 	      clock interrupts.
 8-11  8-11    PSW Key used for complex memory protection mechanism not used under linux
 12      12    1 on s/390 0 on z/Architecture
 13      13    Machine Check Mask 1=enable machine check interrupts
 14      14    Wait State set this to 1 to stop the processor except for interrupts & give 
 	      time to other LPARS used in CPU idle in the kernel to increase overall 
 	      usage of processor resources.
 15      15    Problem state ( if set to 1 certain instructions are disabled )
 	      all linux user programs run with this bit 1 
 	      ( useful info for debugging under VM ).
 16-17 16-17   Address Space Control
 	      00 Primary Space Mode when DAT on
 	      The linux kernel currently runs in this mode, CR1 is affiliated with 
              this mode & points to the primary segment table origin etc.
 	      01 Access register mode this mode is used in functions to 
 	      copy data between kernel & user space.
 	      10 Secondary space mode not used in linux however CR7 the
 	      register affiliated with this mode is & this & normally
 	      CR13=CR7 to allow us to copy data between kernel & user space.
 	      We do this as follows:
 	      We set ar2 to 0 to designate its
 	      affiliated gpr ( gpr2 )to point to primary=kernel space.
 	      We set ar4 to 1 to designate its
 	      affiliated gpr ( gpr4 ) to point to secondary=home=user space
 	      & then essentially do a memcopy(gpr2,gpr4,size) to
 	      copy data between the address spaces, the reason we use home space for the
 	      kernel & don't keep secondary space free is that code will not run in 
 	      secondary space.
 	      11 Home Space Mode all user programs run in this mode.
 	      it is affiliated with CR13.
 18-19 18-19   Condition codes (CC)
 20    20      Fixed point overflow mask if 1=FPU exceptions for this event 
              occur ( normally 0 ) 
 21    21      Decimal overflow mask if 1=FPU exceptions for this event occur 
              ( normally 0 )
 22    22      Exponent underflow mask if 1=FPU exceptions for this event occur 
              ( normally 0 )
 23    23      Significance Mask if 1=FPU exceptions for this event occur 
              ( normally 0 )
 24-31 24-30   Reserved Must be 0.
      31      Extended Addressing Mode
      32      Basic Addressing Mode
              Used to set addressing mode
 	      PSW 31   PSW 32
                0         0        24 bit
                0         1        31 bit
                1         1        64 bit
 32             1=31 bit addressing mode 0=24 bit addressing mode (for backward 
               compatibility), linux always runs with this bit set to 1
 33-64          Instruction address.
      33-63    Reserved must be 0
      64-127   Address
               In 24 bits mode bits 64-103=0 bits 104-127 Address 
               In 31 bits mode bits 64-96=0 bits 97-127 Address
               Note: unlike 31 bit mode on s/390 bit 96 must be zero
 	       when loading the address with LPSWE otherwise a 
               specification exception occurs, LPSW is fully backward
               compatible.
 Prefix Page(s)
 --------------	  
 This per cpu memory area is too intimately tied to the processor not to mention.
 It exists between the real addresses 0-4096 on s/390 & 0-8192 z/Architecture & is exchanged 
 with a 1 page on s/390 or 2 pages on z/Architecture in absolute storage by the set 
 prefix instruction in linux'es startup. 
 This page is mapped to a different prefix for each processor in an SMP configuration
 ( assuming the os designer is sane of course :-) ).
 Bytes 0-512 ( 200 hex ) on s/390 & 0-512,4096-4544,4604-5119 currently on z/Architecture 
 are used by the processor itself for holding such information as exception indications & 
 entry points for exceptions.
 Bytes after 0xc00 hex are used by linux for per processor globals on s/390 & z/Architecture 
 ( there is a gap on z/Architecture too currently between 0xc00 & 1000 which linux uses ).
 The closest thing to this on traditional architectures is the interrupt
 vector table. This is a good thing & does simplify some of the kernel coding
 however it means that we now cannot catch stray NULL pointers in the
 kernel without hard coded checks.
 Address Spaces on Intel Linux
 =============================
 The traditional Intel Linux is approximately mapped as follows forgive
 the ascii art.
 0xFFFFFFFF 4GB Himem                        *****************
                                            *               *
                                            * Kernel Space  *
                                            *               *
                                            *****************          ****************
 User Space Himem (typically 0xC0000000 3GB )*  User Stack   *          *              *
 				            *****************          *              *
 					    *  Shared Libs  *          * Next Process *          
                                            *****************          *     to       *  
 					    *               *    <==   *     Run      *  <==
 					    *  User Program *          *              *
 					    *   Data BSS    *          *              *
                                            *	 Text       *          *              *
         			            *   Sections    *          *              *
 0x00000000         			    *****************          ****************
 Now it is easy to see that on Intel it is quite easy to recognise a kernel address 
 as being one greater than user space himem ( in this case 0xC0000000).
 & addresses of less than this are the ones in the current running program on this
 processor ( if an smp box ).
 If using the virtual machine ( VM ) as a debugger it is quite difficult to
 know which user process is running as the address space you are looking at
 could be from any process in the run queue.
 The limitation of Intels addressing technique is that the linux
 kernel uses a very simple real address to virtual addressing technique
 of Real Address=Virtual Address-User Space Himem.
 This means that on Intel the kernel linux can typically only address
 Himem=0xFFFFFFFF-0xC0000000=1GB & this is all the RAM these machines
 can typically use.
 They can lower User Himem to 2GB or lower & thus be
 able to use 2GB of RAM however this shrinks the maximum size
 of User Space from 3GB to 2GB they have a no win limit of 4GB unless
 they go to 64 Bit.
 On 390 our limitations & strengths make us slightly different.
 For backward compatibility we are only allowed use 31 bits (2GB)
 of our 32 bit addresses, however, we use entirely separate address 
 spaces for the user & kernel.
 This means we can support 2GB of non Extended RAM on s/390, & more
 with the Extended memory management swap device & 
 currently 4TB of physical memory currently on z/Architecture.
 Address Spaces on Linux for s/390 & z/Architecture
 ==================================================
 Our addressing scheme is as follows
 Himem 0x7fffffff 2GB on s/390    *****************          ****************
 currently 0x3ffffffffff (2^42)-1 *  User Stack   *          *              *
 on z/Architecture.		 *****************          *              *
 		                 *  Shared Libs  *          *              *      
                                 *****************          *              *  
 			         *               *          *    Kernel    *  
 		                 *  User Program *          *              *
 		                 *   Data BSS    *          *              *
                                 *    Text       *          *              *
            			 *   Sections    *          *              *
 0x00000000                       *****************          ****************
 This also means that we need to look at the PSW problem state bit
 or the addressing mode to decide whether we are looking at
 user or kernel space.
 Virtual Addresses on s/390 & z/Architecture
 ===========================================
 A virtual address on s/390 is made up of 3 parts
 The SX ( segment index, roughly corresponding to the PGD & PMD in linux terminology ) 
 being bits 1-11.
 The PX ( page index, corresponding to the page table entry (pte) in linux terminology )
 being bits 12-19. 
 The remaining bits BX (the byte index are the offset in the page )
 i.e. bits 20 to 31.
 On z/Architecture in linux we currently make up an address from 4 parts.
 The region index bits (RX) 0-32 we currently use bits 22-32
 The segment index (SX) being bits 33-43
 The page index (PX) being bits  44-51
 The byte index (BX) being bits  52-63
 Notes:
 1) s/390 has no PMD so the PMD is really the PGD also.
 A lot of this stuff is defined in pgtable.h.
 2) Also seeing as s/390's page indexes are only 1k  in size 
 (bits 12-19 x 4 bytes per pte ) we use 1 ( page 4k )
 to make the best use of memory by updating 4 segment indices 
 entries each time we mess with a PMD & use offsets 
 0,1024,2048 & 3072 in this page as for our segment indexes.
 On z/Architecture our page indexes are now 2k in size
 ( bits 12-19 x 8 bytes per pte ) we do a similar trick
 but only mess with 2 segment indices each time we mess with
 a PMD.
 3) As z/Architecture supports upto a massive 5-level page table lookup we 
 can only use 3 currently on Linux ( as this is all the generic kernel
 currently supports ) however this may change in future
 this allows us to access ( according to my sums )
 4TB of virtual storage per process i.e.
 4096*512(PTES)*1024(PMDS)*2048(PGD) = 4398046511104 bytes,
 enough for another 2 or 3 of years I think :-).
 to do this we use a region-third-table designation type in
 our address space control registers.
 The Linux for s/390 & z/Architecture Kernel Task Structure
 ==========================================================
 Each process/thread under Linux for S390 has its own kernel task_struct
 defined in linux/include/linux/sched.h
 The S390 on initialisation & resuming of a process on a cpu sets
 the __LC_KERNEL_STACK variable in the spare prefix area for this cpu
 (which we use for per-processor globals).
 The kernel stack pointer is intimately tied with the task structure for
 each processor as follows.
                      s/390
            ************************
            *  1 page kernel stack *
 	    *        ( 4K )        *
            ************************
            *   1 page task_struct *        
            *        ( 4K )        *
 8K aligned  ************************ 
                 z/Architecture
            ************************
            *  2 page kernel stack *
 	    *        ( 8K )        *
            ************************
            *  2 page task_struct  *        
            *        ( 8K )        *
 16K aligned ************************ 
 What this means is that we don't need to dedicate any register or global variable
 to point to the current running process & can retrieve it with the following
 very simple construct for s/390 & one very similar for z/Architecture.
 static inline struct task_struct * get_current(void)
 {
        struct task_struct *current;
        __asm__("lhi   %0,-8192\n\t"
                "nr    %0,15"
                : "=r" (current) );
        return current;
 }
 i.e. just anding the current kernel stack pointer with the mask -8192.
 Thankfully because Linux doesn't have support for nested IO interrupts
 & our devices have large buffers can survive interrupts being shut for 
 short amounts of time we don't need a separate stack for interrupts.
 Register Usage & Stackframes on Linux for s/390 & z/Architecture
 =================================================================
 Overview:
 ---------
 This is the code that gcc produces at the top & the bottom of
 each function. It usually is fairly consistent & similar from 
 function to function & if you know its layout you can probably
 make some headway in finding the ultimate cause of a problem
 after a crash without a source level debugger.
 Note: To follow stackframes requires a knowledge of C or Pascal &
 limited knowledge of one assembly language.
 It should be noted that there are some differences between the
 s/390 & z/Architecture stack layouts as the z/Architecture stack layout didn't have
 to maintain compatibility with older linkage formats.
 Glossary:
 ---------
 alloca:
 This is a built in compiler function for runtime allocation
 of extra space on the callers stack which is obviously freed
 up on function exit ( e.g. the caller may choose to allocate nothing
 of a buffer of 4k if required for temporary purposes ), it generates 
 very efficient code ( a few cycles  ) when compared to alternatives 
 like malloc.
 automatics: These are local variables on the stack,
 i.e they aren't in registers & they aren't static.
 back-chain:
 This is a pointer to the stack pointer before entering a
 framed functions ( see frameless function ) prologue got by 
 dereferencing the address of the current stack pointer,
 i.e. got by accessing the 32 bit value at the stack pointers
 current location.
 base-pointer:
 This is a pointer to the back of the literal pool which
 is an area just behind each procedure used to store constants
 in each function.
 call-clobbered: The caller probably needs to save these registers if there 
 is something of value in them, on the stack or elsewhere before making a 
 call to another procedure so that it can restore it later.
 epilogue:
 The code generated by the compiler to return to the caller.
 frameless-function
 A frameless function in Linux for s390 & z/Architecture is one which doesn't 
 need more than the register save area ( 96 bytes on s/390, 160 on z/Architecture )
 given to it by the caller.
 A frameless function never:
 1) Sets up a back chain.
 2) Calls alloca.
 3) Calls other normal functions
 4) Has automatics.
 GOT-pointer:
 This is a pointer to the global-offset-table in ELF
 ( Executable Linkable Format, Linux'es most common executable format ),
 all globals & shared library objects are found using this pointer.
 lazy-binding
 ELF shared libraries are typically only loaded when routines in the shared
 library are actually first called at runtime. This is lazy binding.
 procedure-linkage-table
 This is a table found from the GOT which contains pointers to routines
 in other shared libraries which can't be called to by easier means.
 prologue:
 The code generated by the compiler to set up the stack frame.
 outgoing-args:
 This is extra area allocated on the stack of the calling function if the
 parameters for the callee's cannot all be put in registers, the same
 area can be reused by each function the caller calls.
 routine-descriptor:
 A COFF  executable format based concept of a procedure reference 
 actually being 8 bytes or more as opposed to a simple pointer to the routine.
 This is typically defined as follows
 Routine Descriptor offset 0=Pointer to Function
 Routine Descriptor offset 4=Pointer to Table of Contents
 The table of contents/TOC is roughly equivalent to a GOT pointer.
 & it means that shared libraries etc. can be shared between several
 environments each with their own TOC.
 static-chain: This is used in nested functions a concept adopted from pascal 
 by gcc not used in ansi C or C++ ( although quite useful ), basically it
 is a pointer used to reference local variables of enclosing functions.
 You might come across this stuff once or twice in your lifetime.
 e.g.
 The function below should return 11 though gcc may get upset & toss warnings 
 about unused variables.
 int FunctionA(int a)
 {
 	int b;
 	FunctionC(int c)
 	{
 		b=c+1;
 	}
 	FunctionC(10);
 	return(b);
 }
 s/390 & z/Architecture Register usage
 =====================================
 r0       used by syscalls/assembly                  call-clobbered
 r1	 used by syscalls/assembly                  call-clobbered
 r2       argument 0 / return value 0                call-clobbered
 r3       argument 1 / return value 1 (if long long) call-clobbered
 r4       argument 2                                 call-clobbered
 r5       argument 3                                 call-clobbered
 r6	 argument 5                                 saved
 r7       pointer-to arguments 5 to ...              saved      
 r8       this & that                                saved
 r9       this & that                                saved
 r10      static-chain ( if nested function )        saved
 r11      frame-pointer ( if function used alloca )  saved
 r12      got-pointer                                saved
 r13      base-pointer                               saved
 r14      return-address                             saved
 r15      stack-pointer                              saved
 f0       argument 0 / return value ( float/double ) call-clobbered
 f2       argument 1                                 call-clobbered
 f4       z/Architecture argument 2                  saved
 f6       z/Architecture argument 3                  saved
 The remaining floating points
 f1,f3,f5 f7-f15 are call-clobbered.
 Notes:
 ------
 1) The only requirement is that registers which are used
 by the callee are saved, e.g. the compiler is perfectly
 capible of using r11 for purposes other than a frame a
 frame pointer if a frame pointer is not needed.
 2) In functions with variable arguments e.g. printf the calling procedure 
 is identical to one without variable arguments & the same number of 
 parameters. However, the prologue of this function is somewhat more
 hairy owing to it having to move these parameters to the stack to
 get va_start, va_arg & va_end to work.
 3) Access registers are currently unused by gcc but are used in
 the kernel. Possibilities exist to use them at the moment for
 temporary storage but it isn't recommended.
 4) Only 4 of the floating point registers are used for
 parameter passing as older machines such as G3 only have only 4
 & it keeps the stack frame compatible with other compilers.
 However with IEEE floating point emulation under linux on the
 older machines you are free to use the other 12.
 5) A long long or double parameter cannot be have the 
 first 4 bytes in a register & the second four bytes in the 
 outgoing args area. It must be purely in the outgoing args
 area if crossing this boundary.
 6) Floating point parameters are mixed with outgoing args
 on the outgoing args area in the order the are passed in as parameters.
 7) Floating point arguments 2 & 3 are saved in the outgoing args area for 
 z/Architecture
 Stack Frame Layout
 ------------------
 s/390     z/Architecture
 0         0             back chain ( a 0 here signifies end of back chain )
 4         8             eos ( end of stack, not used on Linux for S390 used in other linkage formats )
 8         16            glue used in other s/390 linkage formats for saved routine descriptors etc.
 12        24            glue used in other s/390 linkage formats for saved routine descriptors etc.
 16        32            scratch area
 20        40            scratch area
 24        48            saved r6 of caller function
 28        56            saved r7 of caller function
 32        64            saved r8 of caller function
 36        72            saved r9 of caller function
 40        80            saved r10 of caller function
 44        88            saved r11 of caller function
 48        96            saved r12 of caller function
 52        104           saved r13 of caller function
 56        112           saved r14 of caller function
 60        120           saved r15 of caller function
 64        128           saved f4 of caller function
 72        132           saved f6 of caller function
 80                      undefined
 96        160           outgoing args passed from caller to callee
 96+x      160+x         possible stack alignment ( 8 bytes desirable )
 96+x+y    160+x+y       alloca space of caller ( if used )
 96+x+y+z  160+x+y+z     automatics of caller ( if used )
 0                       back-chain
 A sample program with comments.
 ===============================
 Comments on the function test
 -----------------------------
 1) It didn't need to set up a pointer to the constant pool gpr13 as it isn't used
 ( :-( ).
 2) This is a frameless function & no stack is bought.
 3) The compiler was clever enough to recognise that it could return the
 value in r2 as well as use it for the passed in parameter ( :-) ).
 4) The basr ( branch relative & save ) trick works as follows the instruction 
 has a special case with r0,r0 with some instruction operands is understood as 
 the literal value 0, some risc architectures also do this ). So now
 we are branching to the next address & the address new program counter is
 in r13,so now we subtract the size of the function prologue we have executed
 + the size of the literal pool to get to the top of the literal pool
 0040037c int test(int b)
 {                                                          # Function prologue below
  40037c:	90 de f0 34 	stm	%r13,%r14,52(%r15) # Save registers r13 & r14
  400380:	0d d0       	basr	%r13,%r0           # Set up pointer to constant pool using
  400382:	a7 da ff fa 	ahi	%r13,-6            # basr trick
 	return(5+b);
 	                                                   # Huge main program
  400386:	a7 2a 00 05 	ahi	%r2,5              # add 5 to r2
                                                           # Function epilogue below 
  40038a:	98 de f0 34 	lm	%r13,%r14,52(%r15) # restore registers r13 & 14
  40038e:	07 fe       	br	%r14               # return
 }
 Comments on the function main
 -----------------------------
 1) The compiler did this function optimally ( 8-) )
 Literal pool for main.
 400390:	ff ff ff ec 	.long 0xffffffec
 main(int argc,char *argv[])
 {                                                          # Function prologue below
  400394:	90 bf f0 2c 	stm	%r11,%r15,44(%r15) # Save necessary registers
  400398:	18 0f       	lr	%r0,%r15           # copy stack pointer to r0
  40039a:	a7 fa ff a0 	ahi	%r15,-96           # Make area for callee saving 
  40039e:	0d d0       	basr	%r13,%r0           # Set up r13 to point to
  4003a0:	a7 da ff f0 	ahi	%r13,-16           # literal pool
  4003a4:	50 00 f0 00 	st	%r0,0(%r15)        # Save backchain
 	return(test(5));                                   # Main Program Below
  4003a8:	58 e0 d0 00 	l	%r14,0(%r13)       # load relative address of test from
 						           # literal pool
  4003ac:	a7 28 00 05 	lhi	%r2,5              # Set first parameter to 5
  4003b0:	4d ee d0 00 	bas	%r14,0(%r14,%r13)  # jump to test setting r14 as return
 							   # address using branch & save instruction.
 							   # Function Epilogue below
  4003b4:	98 bf f0 8c 	lm	%r11,%r15,140(%r15)# Restore necessary registers.
  4003b8:	07 fe       	br	%r14               # return to do program exit 
 }
 Compiler updates
 ----------------
 main(int argc,char *argv[])
 {
  4004fc:	90 7f f0 1c       	stm	%r7,%r15,28(%r15)
  400500:	a7 d5 00 04       	bras	%r13,400508 <main+0xc>
  400504:	00 40 04 f4       	.long	0x004004f4 
  # compiler now puts constant pool in code to so it saves an instruction 
  400508:	18 0f             	lr	%r0,%r15
  40050a:	a7 fa ff a0       	ahi	%r15,-96
  40050e:	50 00 f0 00       	st	%r0,0(%r15)
 	return(test(5));
  400512:	58 10 d0 00       	l	%r1,0(%r13)
  400516:	a7 28 00 05       	lhi	%r2,5
  40051a:	0d e1             	basr	%r14,%r1
  # compiler adds 1 extra instruction to epilogue this is done to
  # avoid processor pipeline stalls owing to data dependencies on g5 &
  # above as register 14 in the old code was needed directly after being loaded 
  # by the lm	%r11,%r15,140(%r15) for the br %14.
  40051c:	58 40 f0 98       	l	%r4,152(%r15)
  400520:	98 7f f0 7c       	lm	%r7,%r15,124(%r15)
  400524:	07 f4             	br	%r4
 }
 Hartmut ( our compiler developer ) also has been threatening to take out the
 stack backchain in optimised code as this also causes pipeline stalls, you
 have been warned.
 64 bit z/Architecture code disassembly
 --------------------------------------
 If you understand the stuff above you'll understand the stuff
 below too so I'll avoid repeating myself & just say that 
 some of the instructions have g's on the end of them to indicate
 they are 64 bit & the stack offsets are a bigger, 
 the only other difference you'll find between 32 & 64 bit is that
 we now use f4 & f6 for floating point arguments on 64 bit.
 00000000800005b0 <test>:
 int test(int b)
 {
 	return(5+b);
    800005b0:	a7 2a 00 05       	ahi	%r2,5
    800005b4:	b9 14 00 22       	lgfr	%r2,%r2 # downcast to integer
    800005b8:	07 fe             	br	%r14
    800005ba:	07 07             	bcr	0,%r7
 }
 00000000800005bc <main>:
 main(int argc,char *argv[])
 { 
    800005bc:	eb bf f0 58 00 24 	stmg	%r11,%r15,88(%r15)
    800005c2:	b9 04 00 1f       	lgr	%r1,%r15
    800005c6:	a7 fb ff 60       	aghi	%r15,-160
    800005ca:	e3 10 f0 00 00 24 	stg	%r1,0(%r15)
 	return(test(5));
    800005d0:	a7 29 00 05       	lghi	%r2,5
    # brasl allows jumps > 64k & is overkill here bras would do fune
    800005d4:	c0 e5 ff ff ff ee 	brasl	%r14,800005b0 <test> 
    800005da:	e3 40 f1 10 00 04 	lg	%r4,272(%r15)
    800005e0:	eb bf f0 f8 00 04 	lmg	%r11,%r15,248(%r15)
    800005e6:	07 f4             	br	%r4
 }
 Compiling programs for debugging on Linux for s/390 & z/Architecture
 ====================================================================
 -gdwarf-2 now works it should be considered the default debugging
 format for s/390 & z/Architecture as it is more reliable for debugging
 shared libraries,  normal -g debugging works much better now
 Thanks to the IBM java compiler developers bug reports. 
 This is typically done adding/appending the flags -g or -gdwarf-2 to the 
 CFLAGS & LDFLAGS variables Makefile of the program concerned.
 If using gdb & you would like accurate displays of registers &
 stack traces compile without optimisation i.e make sure
 that there is no -O2 or similar on the CFLAGS line of the Makefile &
 the emitted gcc commands, obviously this will produce worse code 
 ( not advisable for shipment ) but it is an  aid to the debugging process.
 This aids debugging because the compiler will copy parameters passed in
 in registers onto the stack so backtracing & looking at passed in
 parameters will work, however some larger programs which use inline functions
 will not compile without optimisation.
 Debugging with optimisation has since much improved after fixing
 some bugs, please make sure you are using gdb-5.0 or later developed 
 after Nov'2000.
 Figuring out gcc compile errors
 ===============================
 If you are getting a lot of syntax errors compiling a program & the problem
 isn't blatantly obvious from the source.
 It often helps to just preprocess the file, this is done with the -E
 option in gcc.
 What this does is that it runs through the very first phase of compilation
 ( compilation in gcc is done in several stages & gcc calls many programs to
 achieve its end result ) with the -E option gcc just calls the gcc preprocessor (cpp).
 The c preprocessor does the following, it joins all the files #included together
 recursively ( #include files can #include other files ) & also the c file you wish to compile.
 It puts a fully qualified path of the #included files in a comment & it
 does macro expansion.
 This is useful for debugging because
 1) You can double check whether the files you expect to be included are the ones
 that are being included ( e.g. double check that you aren't going to the i386 asm directory ).
 2) Check that macro definitions aren't clashing with typedefs,
 3) Check that definitions aren't being used before they are being included.
 4) Helps put the line emitting the error under the microscope if it contains macros.
 For convenience the Linux kernel's makefile will do preprocessing automatically for you
 by suffixing the file you want built with .i ( instead of .o )
 e.g.
 from the linux directory type
 make arch/s390/kernel/signal.i
 this will build
 s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
 -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce   -E arch/s390/kernel/signal.c
 > arch/s390/kernel/signal.i  
 Now look at signal.i you should see something like.
 # 1 "/home1/barrow/linux/include/asm/types.h" 1
 typedef unsigned short umode_t;
 typedef __signed__ char __s8;
 typedef unsigned char __u8;
 typedef __signed__ short __s16;
 typedef unsigned short __u16;
 If instead you are getting errors further down e.g.
 unknown instruction:2515 "move.l" or better still unknown instruction:2515 
 "Fixme not implemented yet, call Martin" you are probably are attempting to compile some code 
 meant for another architecture or code that is simply not implemented, with a fixme statement
 stuck into the inline assembly code so that the author of the file now knows he has work to do.
 To look at the assembly emitted by gcc just before it is about to call gas ( the gnu assembler )
 use the -S option.
 Again for your convenience the Linux kernel's Makefile will hold your hand &
 do all this donkey work for you also by building the file with the .s suffix.
 e.g.
 from the Linux directory type 
 make arch/s390/kernel/signal.s 
 s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
 -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce  -S arch/s390/kernel/signal.c 
 -o arch/s390/kernel/signal.s  
 This will output something like, ( please note the constant pool & the useful comments
 in the prologue to give you a hand at interpreting it ).
 .LC54:
 	.string	"misaligned (__u16 *) in __xchg\n"
 .LC57:
 	.string	"misaligned (__u32 *) in __xchg\n"
 .L$PG1: # Pool sys_sigsuspend
 .LC192:
 	.long	-262401
 .LC193:
 	.long	-1
 .LC194:
 	.long	schedule-.L$PG1
 .LC195:
 	.long	do_signal-.L$PG1
 	.align 4
 .globl sys_sigsuspend
 	.type	 sys_sigsuspend,@function
 sys_sigsuspend:
 #	leaf function           0
 #	automatics              16
 #	outgoing args           0
 #	need frame pointer      0
 #	call alloca             0
 #	has varargs             0
 #	incoming args (stack)   0
 #	function length         168
 	STM	8,15,32(15)
 	LR	0,15
 	AHI	15,-112
 	BASR	13,0
 .L$CO1:	AHI	13,.L$PG1-.L$CO1
 	ST	0,0(15)
 	LR    8,2
 	N     5,.LC192-.L$PG1(13) 
 Adding -g to the above output makes the output even more useful
 e.g. typing
 make CC:="s390-gcc -g" kernel/sched.s
 which compiles.
 s390-gcc -g -D__KERNEL__ -I/home/barrow/linux-2.3/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce   -S kernel/sched.c -o kernel/sched.s 
 also outputs stabs ( debugger ) info, from this info you can find out the
 offsets & sizes of various elements in structures.
 e.g. the stab for the structure
 struct rlimit {
 	unsigned long	rlim_cur;
 	unsigned long	rlim_max;
 };
 is
 .stabs "rlimit:T(151,2)=s8rlim_cur:(0,5),0,32;rlim_max:(0,5),32,32;;",128,0,0,0
 from this stab you can see that 
 rlimit_cur starts at bit offset 0 & is 32 bits in size
 rlimit_max starts at bit offset 32 & is 32 bits in size.
 Debugging Tools:
 ================
 objdump
 =======
 This is a tool with many options the most useful being ( if compiled with -g).
 objdump --source <victim program or object file> > <victims debug listing >
 The whole kernel can be compiled like this ( Doing this will make a 17MB kernel
 & a 200 MB listing ) however you have to strip it before building the image
 using the strip command to make it a more reasonable size to boot it.
 A source/assembly mixed dump of the kernel can be done with the line
 objdump --source vmlinux > vmlinux.lst
 Also, if the file isn't compiled -g, this will output as much debugging information
 as it can (e.g. function names). This is very slow as it spends lots
 of time searching for debugging info. The following self explanatory line should be used 
 instead if the code isn't compiled -g, as it is much faster:
 objdump --disassemble-all --syms vmlinux > vmlinux.lst  
 As hard drive space is valuble most of us use the following approach.
 1) Look at the emitted psw on the console to find the crash address in the kernel.
 2) Look at the file System.map ( in the linux directory ) produced when building 
 the kernel to find the closest address less than the current PSW to find the
 offending function.
 3) use grep or similar to search the source tree looking for the source file
 with this function if you don't know where it is.
 4) rebuild this object file with -g on, as an example suppose the file was
 ( /arch/s390/kernel/signal.o ) 
 5) Assuming the file with the erroneous function is signal.c Move to the base of the 
 Linux source tree.
 6) rm /arch/s390/kernel/signal.o
 7) make /arch/s390/kernel/signal.o
 8) watch the gcc command line emitted
 9) type it in again or alternatively cut & paste it on the console adding the -g option.
 10) objdump --source arch/s390/kernel/signal.o > signal.lst
 This will output the source & the assembly intermixed, as the snippet below shows
 This will unfortunately output addresses which aren't the same
 as the kernel ones you should be able to get around the mental arithmetic
 by playing with the --adjust-vma parameter to objdump.
 static inline void spin_lock(spinlock_t *lp)
 {
      a0:       18 34           lr      %r3,%r4
      a2:       a7 3a 03 bc     ahi     %r3,956
        __asm__ __volatile("    lhi   1,-1\n"
      a6:       a7 18 ff ff     lhi     %r1,-1
      aa:       1f 00           slr     %r0,%r0
      ac:       ba 01 30 00     cs      %r0,%r1,0(%r3)
      b0:       a7 44 ff fd     jm      aa <sys_sigsuspend+0x2e>
        saveset = current->blocked;
      b4:       d2 07 f0 68     mvc     104(8,%r15),972(%r4)
      b8:       43 cc
        return (set->sig[0] & mask) != 0;
 } 
 6) If debugging under VM go down to that section in the document for more info.
 I now have a tool which takes the pain out of --adjust-vma
 & you are able to do something like
 make /arch/s390/kernel/traps.lst
 & it automatically generates the correctly relocated entries for
 the text segment in traps.lst.
 This tool is now standard in linux distro's in scripts/makelst
 strace:
 -------
 Q. What is it ?
 A. It is a tool for intercepting calls to the kernel & logging them
 to a file & on the screen.
 Q. What use is it ?
 A. You can used it to find out what files a particular program opens.
 Example 1
 ---------
 If you wanted to know does ping work but didn't have the source 
 strace ping -c 1 127.0.0.1  
 & then look at the man pages for each of the syscalls below,
 ( In fact this is sometimes easier than looking at some spagetti
 source which conditionally compiles for several architectures ).
 Not everything that it throws out needs to make sense immediately.
 Just looking quickly you can see that it is making up a RAW socket
 for the ICMP protocol.
 Doing an alarm(10) for a 10 second timeout
 & doing a gettimeofday call before & after each read to see 
 how long the replies took, & writing some text to stdout so the user
 has an idea what is going on.
 socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3
 getuid()                                = 0
 setuid(0)                               = 0
 stat("/usr/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory)
 stat("/usr/share/locale/libc/C", 0xbffff134) = -1 ENOENT (No such file or directory)
 stat("/usr/local/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory)
 getpid()                                = 353
 setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [49152], 4) = 0
 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(3, 1), ...}) = 0
 mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40008000
 ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0
 write(1, "PING 127.0.0.1 (127.0.0.1): 56 d"..., 42PING 127.0.0.1 (127.0.0.1): 56 data bytes
 ) = 42
 sigaction(SIGINT, {0x8049ba0, [], SA_RESTART}, {SIG_DFL}) = 0 
 sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {SIG_DFL}) = 0
 gettimeofday({948904719, 138951}, NULL) = 0
 sendto(3, "\10\0D\201a\1\0\0\17#\2178\307\36"..., 64, 0, {sin_family=AF_INET,
 sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, 16) = 64
 sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0
 sigaction(SIGALRM, {0x8049ba0, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0
 alarm(10)                               = 0
 recvfrom(3, "E\0\0T\0005\0\0@\1|r\177\0\0\1\177"..., 192, 0, 
 {sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84
 gettimeofday({948904719, 160224}, NULL) = 0
 recvfrom(3, "E\0\0T\0006\0\0\377\1\275p\177\0"..., 192, 0, 
 {sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84
 gettimeofday({948904719, 166952}, NULL) = 0
 write(1, "64 bytes from 127.0.0.1: icmp_se"..., 
 5764 bytes from 127.0.0.1: icmp_seq=0 ttl=255 time=28.0 ms
 Example 2
 ---------
 strace passwd 2>&1 | grep open
 produces the following output
 open("/etc/ld.so.cache", O_RDONLY)      = 3
 open("/opt/kde/lib/libc.so.5", O_RDONLY) = -1 ENOENT (No such file or directory)
 open("/lib/libc.so.5", O_RDONLY)        = 3
 open("/dev", O_RDONLY)                  = 3
 open("/var/run/utmp", O_RDONLY)         = 3
 open("/etc/passwd", O_RDONLY)           = 3
 open("/etc/shadow", O_RDONLY)           = 3
 open("/etc/login.defs", O_RDONLY)       = 4
 open("/dev/tty", O_RDONLY)              = 4 
 The 2>&1 is done to redirect stderr to stdout & grep is then filtering this input 
 through the pipe for each line containing the string open.
 Example 3
 ---------
 Getting sophisticated
 telnetd crashes & I don't know why
 Steps
 -----
 1) Replace the following line in /etc/inetd.conf
 telnet  stream  tcp     nowait  root    /usr/sbin/in.telnetd -h 
 with
 telnet  stream  tcp     nowait  root    /blah
 2) Create the file /blah with the following contents to start tracing telnetd 
 #!/bin/bash
 /usr/bin/strace -o/t1 -f /usr/sbin/in.telnetd -h 
 3) chmod 700 /blah to make it executable only to root
 4)
 killall -HUP inetd
 or ps aux | grep inetd
 get inetd's process id
 & kill -HUP inetd to restart it.
 Important options
 -----------------
 -o is used to tell strace to output to a file in our case t1 in the root directory
 -f is to follow children i.e.
 e.g in our case above telnetd will start the login process & subsequently a shell like bash.
 You will be able to tell which is which from the process ID's listed on the left hand side
 of the strace output.
 -p<pid> will tell strace to attach to a running process, yup this can be done provided
 it isn't being traced or debugged already & you have enough privileges,
 the reason 2 processes cannot trace or debug the same program is that strace
 becomes the parent process of the one being debugged & processes ( unlike people )
 can have only one parent.
 However the file /t1 will get big quite quickly
 to test it telnet 127.0.0.1
 now look at what files in.telnetd execve'd
 413   execve("/usr/sbin/in.telnetd", ["/usr/sbin/in.telnetd", "-h"], [/* 17 vars */]) = 0
 414   execve("/bin/login", ["/bin/login", "-h", "localhost", "-p"], [/* 2 vars */]) = 0 
 Whey it worked!.
 Other hints:
 ------------
 If the program is not very interactive ( i.e. not much keyboard input )
 & is crashing in one architecture but not in another you can do 
 an strace of both programs under as identical a scenario as you can
 on both architectures outputting to a file then.
 do a diff of the two traces using the diff program
 i.e.
 diff output1 output2
 & maybe you'll be able to see where the call paths differed, this
 is possibly near the cause of the crash. 
 More info
 ---------
 Look at man pages for strace & the various syscalls
 e.g. man strace, man alarm, man socket.
 Performance Debugging
 =====================
 gcc is capible of compiling in profiling code just add the -p option
 to the CFLAGS, this obviously affects program size & performance.
 This can be used by the gprof gnu profiling tool or the
 gcov the gnu code coverage tool ( code coverage is a means of testing
 code quality by checking if all the code in an executable in exercised by
 a tester ).
 Using top to find out where processes are sleeping in the kernel
 ----------------------------------------------------------------
 To do this copy the System.map from the root directory where
 the linux kernel was built to the /boot directory on your 
 linux machine.
 Start top
 Now type fU<return>
 You should see a new field called WCHAN which
 tells you where each process is sleeping here is a typical output.
 6:59pm  up 41 min,  1 user,  load average: 0.00, 0.00, 0.00
 28 processes: 27 sleeping, 1 running, 0 zombie, 0 stopped
 CPU states:  0.0% user,  0.1% system,  0.0% nice, 99.8% idle
 Mem:   254900K av,   45976K used,  208924K free,       0K shrd,   28636K buff
 Swap:       0K av,       0K used,       0K free                    8620K cached
  PID USER     PRI  NI  SIZE  RSS SHARE WCHAN     STAT  LIB %CPU %MEM   TIME COMMAND
  750 root      12   0   848  848   700 do_select S       0  0.1  0.3   0:00 in.telnetd
  767 root      16   0  1140 1140   964           R       0  0.1  0.4   0:00 top
    1 root       8   0   212  212   180 do_select S       0  0.0  0.0   0:00 init
    2 root       9   0     0    0     0 down_inte SW      0  0.0  0.0   0:00 kmcheck
 The time command
 ----------------
 Another related command is the time command which gives you an indication
 of where a process is spending the majority of its time.
 e.g.
 time ping -c 5 nc
 outputs
 real	0m4.054s
 user	0m0.010s
 sys	0m0.010s
 Debugging under VM
 ==================
 Notes
 -----
 Addresses & values in the VM debugger are always hex never decimal
 Address ranges are of the format <HexValue1>-<HexValue2> or <HexValue1>.<HexValue2> 
-e.g. The address range  0x2000 to 0x3000 can be described described as
+e.g. The address range  0x2000 to 0x3000 can be described as 2000-3000 or 2000.1000
-2000-3000 or 2000.1000
 The VM Debugger is case insensitive.
 VM's strengths are usually other debuggers weaknesses you can get at any resource
 no matter how sensitive e.g. memory management resources,change address translation
 in the PSW. For kernel hacking you will reap dividends if you get good at it.
 The VM Debugger displays operators but not operands, probably because some
 of it was written when memory was expensive & the programmer was probably proud that
 it fitted into 2k of memory & the programmers & didn't want to shock hardcore VM'ers by
 changing the interface :-), also the debugger displays useful information on the same line & 
 the author of the code probably felt that it was a good idea not to go over 
 the 80 columns on the screen. 
 As some of you are probably in a panic now this isn't as unintuitive as it may seem
 as the 390 instructions are easy to decode mentally & you can make a good guess at a lot 
 of them as all the operands are nibble ( half byte aligned ) & if you have an objdump listing
 also it is quite easy to follow, if you don't have an objdump listing keep a copy of
 the s/390 Reference Summary & look at between pages 2 & 7 or alternatively the
 s/390 principles of operation.
 e.g. even I can guess that 
 0001AFF8' LR    180F        CC 0
 is a ( load register ) lr r0,r15 
 Also it is very easy to tell the length of a 390 instruction from the 2 most significant
 bits in the instruction ( not that this info is really useful except if you are trying to
 make sense of a hexdump of code ).
 Here is a table
 Bits                    Instruction Length
 ------------------------------------------
 00                          2 Bytes
 01                          4 Bytes
 10                          4 Bytes
 11                          6 Bytes
 The debugger also displays other useful info on the same line such as the
 addresses being operated on destination addresses of branches & condition codes.
 e.g.  
 00019736' AHI   A7DAFF0E    CC 1
 000198BA' BRC   A7840004 -> 000198C2'   CC 0
 000198CE' STM   900EF068 >> 0FA95E78    CC 2
 Useful VM debugger commands
 ---------------------------
 I suppose I'd better mention this before I start
 to list the current active traces do 
 Q TR
 there can be a maximum of 255 of these per set
 ( more about trace sets later ).
 To stop traces issue a
 TR END.
 To delete a particular breakpoint issue
 TR DEL <breakpoint number>
 The PA1 key drops to CP mode so you can issue debugger commands,
 Doing alt c (on my 3270 console at least ) clears the screen. 
 hitting b <enter> comes back to the running operating system
 from cp mode ( in our case linux ).
 It is typically useful to add shortcuts to your profile.exec file
 if you have one ( this is roughly equivalent to autoexec.bat in DOS ).
 file here are a few from mine.
 /* this gives me command history on issuing f12 */
 set pf12 retrieve 
 /* this continues */
 set pf8 imm b
 /* goes to trace set a */
 set pf1 imm tr goto a
 /* goes to trace set b */
 set pf2 imm tr goto b
 /* goes to trace set c */
 set pf3 imm tr goto c
 Instruction Tracing
 -------------------
 Setting a simple breakpoint
 TR I PSWA <address>
 To debug a particular function try
 TR I R <function address range>
 TR I on its own will single step.
 TR I DATA <MNEMONIC> <OPTIONAL RANGE> will trace for particular mnemonics
 e.g.
 TR I DATA 4D R 0197BC.4000
 will trace for BAS'es ( opcode 4D ) in the range 0197BC.4000
 if you were inclined you could add traces for all branch instructions &
 suffix them with the run prefix so you would have a backtrace on screen 
 when a program crashes.
 TR BR <INTO OR FROM> will trace branches into or out of an address.
 e.g.
 TR BR INTO 0 is often quite useful if a program is getting awkward & deciding
 to branch to 0 & crashing as this will stop at the address before in jumps to 0.
 TR I R <address range> RUN cmd d g
 single steps a range of addresses but stays running &
 displays the gprs on each step.
 Displaying & modifying Registers
 --------------------------------
 D G will display all the gprs
 Adding a extra G to all the commands is necessary to access the full 64 bit 
 content in VM on z/Architecture obviously this isn't required for access registers
 as these are still 32 bit.
 e.g. DGG instead of DG 
 D X will display all the control registers
 D AR will display all the access registers
 D AR4-7 will display access registers 4 to 7
 CPU ALL D G will display the GRPS of all CPUS in the configuration
 D PSW will display the current PSW
 st PSW 2000 will put the value 2000 into the PSW &
 cause crash your machine.
 D PREFIX displays the prefix offset
 Displaying Memory
 -----------------
 To display memory mapped using the current PSW's mapping try
 D <range>
 To make VM display a message each time it hits a particular address & continue try
 D I<range> will disassemble/display a range of instructions.
 ST addr 32 bit word will store a 32 bit aligned address
 D T<range> will display the EBCDIC in an address ( if you are that way inclined )
 D R<range> will display real addresses ( without DAT ) but with prefixing.
 There are other complex options to display if you need to get at say home space
 but are in primary space the easiest thing to do is to temporarily
 modify the PSW to the other addressing mode, display the stuff & then
 restore it.
 Hints
 -----
 If you want to issue a debugger command without halting your virtual machine with the
 PA1 key try prefixing the command with #CP e.g.
 #cp tr i pswa 2000
 also suffixing most debugger commands with RUN will cause them not
 to stop just display the mnemonic at the current instruction on the console.
 If you have several breakpoints you want to put into your program &
 you get fed up of cross referencing with System.map
 you can do the following trick for several symbols.
 grep do_signal System.map 
 which emits the following among other things
 0001f4e0 T do_signal 
 now you can do
 TR I PSWA 0001f4e0 cmd msg * do_signal
 This sends a message to your own console each time do_signal is entered.
 ( As an aside I wrote a perl script once which automatically generated a REXX
 script with breakpoints on every kernel procedure, this isn't a good idea
 because there are thousands of these routines & VM can only set 255 breakpoints
 at a time so you nearly had to spend as long pruning the file down as you would 
 entering the msg's by hand ),however, the trick might be useful for a single object file.
 On linux'es 3270 emulator x3270 there is a very useful option under the file ment
 Save Screens In File this is very good of keeping a copy of traces. 
 From CMS help <command name> will give you online help on a particular command. 
 e.g. 
 HELP DISPLAY
 Also CP has a file called profile.exec which automatically gets called
 on startup of CMS ( like autoexec.bat ), keeping on a DOS analogy session
 CP has a feature similar to doskey, it may be useful for you to
 use profile.exec to define some keystrokes. 
 e.g.
 SET PF9 IMM B
 This does a single step in VM on pressing F8. 
 SET PF10  ^
 This sets up the ^ key.
 which can be used for ^c (ctrl-c),^z (ctrl-z) which can't be typed directly into some 3270 consoles.
 SET PF11 ^-
 This types the starting keystrokes for a sysrq see SysRq below.
 SET PF12 RETRIEVE
 This retrieves command history on pressing F12.
 Sometimes in VM the display is set up to scroll automatically this
 can be very annoying if there are messages you wish to look at
 to stop this do
 TERM MORE 255 255
 This will nearly stop automatic screen updates, however it will
 cause a denial of service if lots of messages go to the 3270 console,
 so it would be foolish to use this as the default on a production machine.
 Tracing particular processes
 ----------------------------
 The kernel's text segment is intentionally at an address in memory that it will
 very seldom collide with text segments of user programs ( thanks Martin ),
 this simplifies debugging the kernel.
 However it is quite common for user processes to have addresses which collide
 this can make debugging a particular process under VM painful under normal
 circumstances as the process may change when doing a 
 TR I R <address range>.
 Thankfully after reading VM's online help I figured out how to debug
 I particular process.
 Your first problem is to find the STD ( segment table designation )
 of the program you wish to debug.
 There are several ways you can do this here are a few
 1) objdump --syms <program to be debugged> | grep main
 To get the address of main in the program.
 tr i pswa <address of main>
 Start the program, if VM drops to CP on what looks like the entry
 point of the main function this is most likely the process you wish to debug.
 Now do a D X13 or D XG13 on z/Architecture.
 On 31 bit the STD is bits 1-19 ( the STO segment table origin ) 
 & 25-31 ( the STL segment table length ) of CR13.
 now type
 TR I R STD <CR13's value> 0.7fffffff
 e.g.
 TR I R STD 8F32E1FF 0.7fffffff
 Another very useful variation is
 TR STORE INTO STD <CR13's value> <address range>
 for finding out when a particular variable changes.
 An alternative way of finding the STD of a currently running process 
 is to do the following, ( this method is more complex but
 could be quite convenient if you aren't updating the kernel much &
 so your kernel structures will stay constant for a reasonable period of
 time ).
 grep task /proc/<pid>/status
 from this you should see something like
 task: 0f160000 ksp: 0f161de8 pt_regs: 0f161f68
 This now gives you a pointer to the task structure.
 Now make CC:="s390-gcc -g" kernel/sched.s
 To get the task_struct stabinfo.
 ( task_struct is defined in include/linux/sched.h ).
 Now we want to look at
 task->active_mm->pgd
 on my machine the active_mm in the task structure stab is
 active_mm:(4,12),672,32
 its offset is 672/8=84=0x54
 the pgd member in the mm_struct stab is
 pgd:(4,6)=*(29,5),96,32
 so its offset is 96/8=12=0xc
 so we'll
 hexdump -s 0xf160054 /dev/mem | more
 i.e. task_struct+active_mm offset
 to look at the active_mm member
 f160054 0fee cc60 0019 e334 0000 0000 0000 0011
 hexdump -s 0x0feecc6c /dev/mem | more
 i.e. active_mm+pgd offset
 feecc6c 0f2c 0000 0000 0001 0000 0001 0000 0010
 we get something like
 now do 
 TR I R STD <pgd|0x7f> 0.7fffffff
 i.e. the 0x7f is added because the pgd only
 gives the page table origin & we need to set the low bits
 to the maximum possible segment table length.
 TR I R STD 0f2c007f 0.7fffffff
 on z/Architecture you'll probably need to do
 TR I R STD <pgd|0x7> 0.ffffffffffffffff
 to set the TableType to 0x1 & the Table length to 3.
 Tracing Program Exceptions
 --------------------------
 If you get a crash which says something like
 illegal operation or specification exception followed by a register dump
 You can restart linux & trace these using the tr prog <range or value> trace option.
 The most common ones you will normally be tracing for is
 1=operation exception
 2=privileged operation exception
 4=protection exception
 5=addressing exception
 6=specification exception
 10=segment translation exception
 11=page translation exception
 The full list of these is on page 22 of the current s/390 Reference Summary.
 e.g.
 tr prog 10 will trace segment translation exceptions.
 tr prog on its own will trace all program interruption codes.
 Trace Sets
 ----------
 On starting VM you are initially in the INITIAL trace set.
 You can do a Q TR to verify this.
 If you have a complex tracing situation where you wish to wait for instance 
 till a driver is open before you start tracing IO, but know in your
 heart that you are going to have to make several runs through the code till you
 have a clue whats going on. 
 What you can do is
 TR I PSWA <Driver open address>
 hit b to continue till breakpoint
 reach the breakpoint
 now do your
 TR GOTO B 
 TR IO 7c08-7c09 inst int run 
 or whatever the IO channels you wish to trace are & hit b
 To got back to the initial trace set do
 TR GOTO INITIAL
 & the TR I PSWA <Driver open address> will be the only active breakpoint again.
 Tracing linux syscalls under VM
 -------------------------------
 Syscalls are implemented on Linux for S390 by the Supervisor call instruction (SVC) there 256 
 possibilities of these as the instruction is made up of a  0xA opcode & the second byte being
 the syscall number. They are traced using the simple command.
 TR SVC  <Optional value or range>
 the syscalls are defined in linux/include/asm-s390/unistd.h
 e.g. to trace all file opens just do
 TR SVC 5 ( as this is the syscall number of open )
 SMP Specific commands
 ---------------------
 To find out how many cpus you have
 Q CPUS displays all the CPU's available to your virtual machine
 To find the cpu that the current cpu VM debugger commands are being directed at do
-Q CPU to change the current cpu cpu VM debugger commands are being directed at do
+Q CPU to change the current cpu VM debugger commands are being directed at do
 CPU <desired cpu no>
 On a SMP guest issue a command to all CPUs try prefixing the command with cpu all.
 To issue a command to a particular cpu try cpu <cpu number> e.g.
 CPU 01 TR I R 2000.3000
 If you are running on a guest with several cpus & you have a IO related problem
 & cannot follow the flow of code but you know it isnt smp related.
 from the bash prompt issue
 shutdown -h now or halt.
 do a Q CPUS to find out how many cpus you have
 detach each one of them from cp except cpu 0 
 by issuing a 
 DETACH CPU 01-(number of cpus in configuration)
 & boot linux again.
 TR SIGP will trace inter processor signal processor instructions.
 DEFINE CPU 01-(number in configuration) 
 will get your guests cpus back.
 Help for displaying ascii textstrings
 -------------------------------------
 On the very latest VM Nucleus'es VM can now display ascii
 ( thanks Neale for the hint ) by doing
 D TX<lowaddr>.<len>
 e.g.
 D TX0.100
 Alternatively
 =============
 Under older VM debuggers ( I love EBDIC too ) you can use this little program I wrote which
 will convert a command line of hex digits to ascii text which can be compiled under linux & 
 you can copy the hex digits from your x3270 terminal to your xterm if you are debugging
 from a linuxbox.
 This is quite useful when looking at a parameter passed in as a text string
 under VM ( unless you are good at decoding ASCII in your head ).
 e.g. consider tracing an open syscall
 TR SVC 5
 We have stopped at a breakpoint
 000151B0' SVC   0A05     -> 0001909A'   CC 0
 D 20.8 to check the SVC old psw in the prefix area & see was it from userspace
 ( for the layout of the prefix area consult P18 of the s/390 390 Reference Summary 
 if you have it available ).
 V00000020  070C2000 800151B2
 The problem state bit wasn't set &  it's also too early in the boot sequence
 for it to be a userspace SVC if it was we would have to temporarily switch the 
 psw to user space addressing so we could get at the first parameter of the open in
 gpr2.
 Next do a 
 D G2
 GPR  2 =  00014CB4
 Now display what gpr2 is pointing to
 D 00014CB4.20
 V00014CB4  2F646576 2F636F6E 736F6C65 00001BF5
 V00014CC4  FC00014C B4001001 E0001000 B8070707
 Now copy the text till the first 00 hex ( which is the end of the string
 to an xterm & do hex2ascii on it.
 hex2ascii 2F646576 2F636F6E 736F6C65 00 
 outputs
 Decoded Hex:=/ d e v / c o n s o l e 0x00 
 We were opening the console device,
 You can compile the code below yourself for practice :-),
 /*
 *    hex2ascii.c
 *    a useful little tool for converting a hexadecimal command line to ascii
 *
 *    Author(s): Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com)
 *    (C) 2000 IBM Deutschland Entwicklung GmbH, IBM Corporation.
 */   
 #include <stdio.h>
 int main(int argc,char *argv[])
 {
  int cnt1,cnt2,len,toggle=0;
  int startcnt=1;
  unsigned char c,hex;
  if(argc>1&&(strcmp(argv[1],"-a")==0))
     startcnt=2;
  printf("Decoded Hex:=");
  for(cnt1=startcnt;cnt1<argc;cnt1++)
  {
    len=strlen(argv[cnt1]);
    for(cnt2=0;cnt2<len;cnt2++)
    {
       c=argv[cnt1][cnt2];
       if(c>='0'&&c<='9')
 	  c=c-'0';
       if(c>='A'&&c<='F')
 	  c=c-'A'+10;
       if(c>='a'&&c<='f')
 	  c=c-'a'+10;
       switch(toggle)
       {
 	  case 0:
 	     hex=c<<4;
 	     toggle=1;
 	  break;
 	  case 1:
 	     hex+=c;
 	     if(hex<32||hex>127)
 	     {
 		if(startcnt==1)
 		   printf("0x%02X ",(int)hex);
 		else
 		   printf(".");
 	     }
 	     else
 	     {
 	       printf("%c",hex);
 	       if(startcnt==1)
 		  printf(" ");
 	     }
 	     toggle=0;
 	  break;
       }
    }
  }
  printf("\n");
 }
 Stack tracing under VM
 ----------------------
 A basic backtrace
 -----------------
 Here are the tricks I use 9 out of 10 times it works pretty well,
 When your backchain reaches a dead end
 --------------------------------------
 This can happen when an exception happens in the kernel & the kernel is entered twice
 if you reach the NULL pointer at the end of the back chain you should be
 able to sniff further back if you follow the following tricks.
 1) A kernel address should be easy to recognise since it is in
 primary space & the problem state bit isn't set & also
 The Hi bit of the address is set.
 2) Another backchain should also be easy to recognise since it is an 
 address pointing to another address approximately 100 bytes or 0x70 hex
 behind the current stackpointer.
 Here is some practice.
 boot the kernel & hit PA1 at some random time
 d g to display the gprs, this should display something like
 GPR  0 =  00000001  00156018  0014359C  00000000
 GPR  4 =  00000001  001B8888  000003E0  00000000
 GPR  8 =  00100080  00100084  00000000  000FE000
 GPR 12 =  00010400  8001B2DC  8001B36A  000FFED8
 Note that GPR14 is a return address but as we are real men we are going to
 trace the stack.
 display 0x40 bytes after the stack pointer.
 V000FFED8  000FFF38 8001B838 80014C8E 000FFF38
 V000FFEE8  00000000 00000000 000003E0 00000000
 V000FFEF8  00100080 00100084 00000000 000FE000
 V000FFF08  00010400 8001B2DC 8001B36A 000FFED8
 Ah now look at whats in sp+56 (sp+0x38) this is 8001B36A our saved r14 if
 you look above at our stackframe & also agrees with GPR14.
 now backchain 
 d 000FFF38.40
 we now are taking the contents of SP to get our first backchain.
 V000FFF38  000FFFA0 00000000 00014995 00147094
 V000FFF48  00147090 001470A0 000003E0 00000000
 V000FFF58  00100080 00100084 00000000 001BF1D0
 V000FFF68  00010400 800149BA 80014CA6 000FFF38
 This displays a 2nd return address of 80014CA6
 now do d 000FFFA0.40 for our 3rd backchain
 V000FFFA0  04B52002 0001107F 00000000 00000000
 V000FFFB0  00000000 00000000 FF000000 0001107F
 V000FFFC0  00000000 00000000 00000000 00000000
 V000FFFD0  00010400 80010802 8001085A 000FFFA0
 our 3rd return address is 8001085A
 as the 04B52002 looks suspiciously like rubbish it is fair to assume that the kernel entry routines
 for the sake of optimisation dont set up a backchain.
 now look at System.map to see if the addresses make any sense.
 grep -i 0001b3 System.map
 outputs among other things
 0001b304 T cpu_idle 
 so 8001B36A
 is cpu_idle+0x66 ( quiet the cpu is asleep, don't wake it )
 grep -i 00014 System.map 
 produces among other things
 00014a78 T start_kernel  
 so 0014CA6 is start_kernel+some hex number I can't add in my head.
 grep -i 00108 System.map 
 this produces
 00010800 T _stext
 so   8001085A is _stext+0x5a
 Congrats you've done your first backchain.
 s/390 & z/Architecture IO Overview
 ==================================
 I am not going to give a course in 390 IO architecture as this would take me quite a
 while & I'm no expert. Instead I'll give a 390 IO architecture summary for Dummies if you have 
 the s/390 principles of operation available read this instead. If nothing else you may find a few 
 useful keywords in here & be able to use them on a web search engine like altavista to find 
 more useful information.
 Unlike other bus architectures modern 390 systems do their IO using mostly
 fibre optics & devices such as tapes & disks can be shared between several mainframes,
 also S390 can support upto 65536 devices while a high end PC based system might be choking 
 with around 64. Here is some of the common IO terminology
 Subchannel:
 This is the logical number most IO commands use to talk to an IO device there can be upto
 0x10000 (65536) of these in a configuration typically there is a few hundred. Under VM
 for simplicity they are allocated contiguously, however on the native hardware they are not
 they typically stay consistent between boots provided no new hardware is inserted or removed.
 Under Linux for 390 we use these as IRQ's & also when issuing an IO command (CLEAR SUBCHANNEL,
 HALT SUBCHANNEL,MODIFY SUBCHANNEL,RESUME SUBCHANNEL,START SUBCHANNEL,STORE SUBCHANNEL & 
 TEST SUBCHANNEL ) we use this as the ID of the device we wish to talk to, the most
 important of these instructions are START SUBCHANNEL ( to start IO ), TEST SUBCHANNEL ( to check
 whether the IO completed successfully ), & HALT SUBCHANNEL ( to kill IO ), a subchannel
 can have up to 8 channel paths to a device this offers redunancy if one is not available.
 Device Number:
 This number remains static & Is closely tied to the hardware, there are 65536 of these
 also they are made up of a CHPID ( Channel Path ID, the most significant 8 bits ) 
 & another lsb 8 bits. These remain static even if more devices are inserted or removed
 from the hardware, there is a 1 to 1 mapping between Subchannels & Device Numbers provided
 devices arent inserted or removed.
 Channel Control Words:
 CCWS are linked lists of instructions initially pointed to by an operation request block (ORB),
 which is initially given to Start Subchannel (SSCH) command along with the subchannel number
 for the IO subsystem to process while the CPU continues executing normal code.
 These come in two flavours, Format 0 ( 24 bit for backward )
 compatibility & Format 1 ( 31 bit ). These are typically used to issue read & write 
 ( & many other instructions ) they consist of a length field & an absolute address field.
 For each IO typically get 1 or 2 interrupts one for channel end ( primary status ) when the
 channel is idle & the second for device end ( secondary status ) sometimes you get both
 concurrently, you check how the IO went on by issuing a TEST SUBCHANNEL at each interrupt,
 from which you receive an Interruption response block (IRB). If you get channel & device end 
 status in the IRB without channel checks etc. your IO probably went okay. If you didn't you
 probably need a doctor to examine the IRB & extended status word etc.
 If an error occurs, more sophistocated control units have a facitity known as
 concurrent sense this means that if an error occurs Extended sense information will
 be presented in the Extended status word in the IRB if not you have to issue a
 subsequent SENSE CCW command after the test subchannel. 
 TPI( Test pending interrupt) can also be used for polled IO but in multitasking multiprocessor
 systems it isn't recommended except for checking special cases ( i.e. non looping checks for
 pending IO etc. ).
 Store Subchannel & Modify Subchannel can be used to examine & modify operating characteristics
 of a subchannel ( e.g. channel paths ).
 Other IO related Terms:
 Sysplex: S390's Clustering Technology
 QDIO: S390's new high speed IO architecture to support devices such as gigabit ethernet,
 this architecture is also designed to be forward compatible with up & coming 64 bit machines.
 General Concepts 
 Input Output Processors (IOP's) are responsible for communicating between
 the mainframe CPU's & the channel & relieve the mainframe CPU's from the
 burden of communicating with IO devices directly, this allows the CPU's to 
 concentrate on data processing. 
 IOP's can use one or more links ( known as channel paths ) to talk to each 
 IO device. It first checks for path availability & chooses an available one,
 then starts ( & sometimes terminates IO ).
 There are two types of channel path: ESCON & the Parallel IO interface.
 IO devices are attached to control units, control units provide the
 logic to interface the channel paths & channel path IO protocols to 
 the IO devices, they can be integrated with the devices or housed separately
 & often talk to several similar devices ( typical examples would be raid 
 controllers or a control unit which connects to 1000 3270 terminals ).
    +---------------------------------------------------------------+
    | +-----+ +-----+ +-----+ +-----+  +----------+  +----------+   |
    | | CPU | | CPU | | CPU | | CPU |  |  Main    |  | Expanded |   |
    | |     | |     | |     | |     |  |  Memory  |  |  Storage |   |
    | +-----+ +-----+ +-----+ +-----+  +----------+  +----------+   | 
    |---------------------------------------------------------------+
    |   IOP        |      IOP      |       IOP                      |
    |---------------------------------------------------------------
    | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | C | 
    ----------------------------------------------------------------
         ||                                              ||
         ||  Bus & Tag Channel Path                      || ESCON
         ||  ======================                      || Channel
         ||  ||                  ||                      || Path
    +----------+               +----------+         +----------+
    |          |               |          |         |          |
    |    CU    |               |    CU    |         |    CU    |
    |          |               |          |         |          |
    +----------+               +----------+         +----------+
       |      |                     |                |       |
 +----------+ +----------+      +----------+   +----------+ +----------+
 |I/O Device| |I/O Device|      |I/O Device|   |I/O Device| |I/O Device|
 +----------+ +----------+      +----------+   +----------+ +----------+
  CPU = Central Processing Unit    
  C = Channel                      
  IOP = IP Processor               
  CU = Control Unit
 The 390 IO systems come in 2 flavours the current 390 machines support both
 The Older 360 & 370 Interface,sometimes called the Parallel I/O interface,
 sometimes called Bus-and Tag & sometimes Original Equipment Manufacturers
 Interface (OEMI).
 This byte wide Parallel channel path/bus has parity & data on the "Bus" cable 
 & control lines on the "Tag" cable. These can operate in byte multiplex mode for
 sharing between several slow devices or burst mode & monopolize the channel for the
 whole burst. Upto 256 devices can be addressed  on one of these cables. These cables are
 about one inch in diameter. The maximum unextended length supported by these cables is
 125 Meters but this can be extended up to 2km with a fibre optic channel extended 
 such as a 3044. The maximum burst speed supported is 4.5 megabytes per second however
 some really old processors support only transfer rates of 3.0, 2.0 & 1.0 MB/sec.
 One of these paths can be daisy chained to up to 8 control units.
 ESCON if fibre optic it is also called FICON 
 Was introduced by IBM in 1990. Has 2 fibre optic cables & uses either leds or lasers
 for communication at a signaling rate of upto 200 megabits/sec. As 10bits are transferred
 for every 8 bits info this drops to 160 megabits/sec & to 18.6 Megabytes/sec once
 control info & CRC are added. ESCON only operates in burst mode.
 ESCONs typical max cable length is 3km for the led version & 20km for the laser version
 known as XDF ( extended distance facility ). This can be further extended by using an
 ESCON director which triples the above mentioned ranges. Unlike Bus & Tag as ESCON is
 serial it uses a packet switching architecture the standard Bus & Tag control protocol
 is however present within the packets. Upto 256 devices can be attached to each control 
 unit that uses one of these interfaces.
 Common 390 Devices include:
 Network adapters typically OSA2,3172's,2116's & OSA-E gigabit ethernet adapters,
 Consoles 3270 & 3215 ( a teletype emulated under linux for a line mode console ).
 DASD's direct access storage devices ( otherwise known as hard disks ).
 Tape Drives.
 CTC ( Channel to Channel Adapters ),
 ESCON or Parallel Cables used as a very high speed serial link
 between 2 machines. We use 2 cables under linux to do a bi-directional serial link.
 Debugging IO on s/390 & z/Architecture under VM
 ===============================================
 Now we are ready to go on with IO tracing commands under VM
 A few self explanatory queries:
 Q OSA
 Q CTC
 Q DISK ( This command is CMS specific )
 Q DASD
 Q OSA on my machine returns
 OSA  7C08 ON OSA   7C08 SUBCHANNEL = 0000
 OSA  7C09 ON OSA   7C09 SUBCHANNEL = 0001
 OSA  7C14 ON OSA   7C14 SUBCHANNEL = 0002
 OSA  7C15 ON OSA   7C15 SUBCHANNEL = 0003
 If you have a guest with certain privileges you may be able to see devices
 which don't belong to you. To avoid this, add the option V.
 e.g.
 Q V OSA
 Now using the device numbers returned by this command we will
 Trace the io starting up on the first device 7c08 & 7c09
 In our simplest case we can trace the 
 start subchannels
 like TR SSCH 7C08-7C09
 or the halt subchannels
 or TR HSCH 7C08-7C09
 MSCH's ,STSCH's I think you can guess the rest
 Ingo's favourite trick is tracing all the IO's & CCWS & spooling them into the reader of another
 VM guest so he can ftp the logfile back to his own machine.I'll do a small bit of this & give you
 a look at the output.
 1) Spool stdout to VM reader
 SP PRT TO (another vm guest ) or * for the local vm guest
 2) Fill the reader with the trace
 TR IO 7c08-7c09 INST INT CCW PRT RUN
 3) Start up linux 
 i 00c  
 4) Finish the trace
 TR END
 5) close the reader
 C PRT
 6) list reader contents
 RDRLIST
 7) copy it to linux4's minidisk 
 RECEIVE / LOG TXT A1 ( replace
 8)
 filel & press F11 to look at it
 You should see something like:
 00020942' SSCH  B2334000    0048813C    CC 0    SCH 0000    DEV 7C08
          CPA 000FFDF0   PARM 00E2C9C4    KEY 0  FPI C0  LPM 80
          CCW    000FFDF0  E4200100 00487FE8   0000  E4240100 ........
          IDAL                                      43D8AFE8
          IDAL                                      0FB76000
 00020B0A'   I/O DEV 7C08 -> 000197BC'   SCH 0000   PARM 00E2C9C4
 00021628' TSCH  B2354000 >> 00488164    CC 0    SCH 0000    DEV 7C08
          CCWA 000FFDF8   DEV STS 0C  SCH STS 00  CNT 00EC
           KEY 0   FPI C0  CC 0   CTLS 4007
 00022238' STSCH B2344000 >> 00488108    CC 0    SCH 0000    DEV 7C08
 If you don't like messing up your readed ( because you possibly booted from it )
 you can alternatively spool it to another readers guest.
 Other common VM device related commands
 ---------------------------------------------
 These commands are listed only because they have
 been of use to me in the past & may be of use to
 you too. For more complete info on each of the commands
 use type HELP <command> from CMS.
 detaching devices
 DET <devno range>
 ATT <devno range> <guest> 
 attach a device to guest * for your own guest
 READY <devno> cause VM to issue a fake interrupt.
 The VARY command is normally only available to VM administrators.
 VARY ON PATH <path> TO <devno range>
 VARY OFF PATH <PATH> FROM <devno range>
 This is used to switch on or off channel paths to devices.
 Q CHPID <channel path ID>
 This displays state of devices using this channel path
 D SCHIB <subchannel>
 This displays the subchannel information SCHIB block for the device.
 this I believe is also only available to administrators.
 DEFINE CTC <devno>
 defines a virtual CTC channel to channel connection
 2 need to be defined on each guest for the CTC driver to use.
 COUPLE  devno userid remote devno
 Joins a local virtual device to a remote virtual device
 ( commonly used for the CTC driver ).
 Building a VM ramdisk under CMS which linux can use
 def vfb-<blocksize> <subchannel> <number blocks>
 blocksize is commonly 4096 for linux.
 Formatting it
 format <subchannel> <driver letter e.g. x> (blksize <blocksize>
 Sharing a disk between multiple guests
 LINK userid devno1 devno2 mode password
 GDB on S390
 ===========
 N.B. if compiling for debugging gdb works better without optimisation 
 ( see Compiling programs for debugging )
 invocation
 ----------
 gdb <victim program> <optional corefile>
 Online help
 -----------
 help: gives help on commands
 e.g.
 help
 help display
 Note gdb's online help is very good use it.
 Assembly
 --------
 info registers: displays registers other than floating point.
 info all-registers: displays floating points as well.
 disassemble: disassembles
 e.g.
 disassemble without parameters will disassemble the current function
 disassemble $pc $pc+10 
 Viewing & modifying variables
 -----------------------------
 print or p: displays variable or register
 e.g. p/x $sp will display the stack pointer
 display: prints variable or register each time program stops
 e.g.
 display/x $pc will display the program counter
 display argc
 undisplay : undo's display's
 info breakpoints: shows all current breakpoints
 info stack: shows stack back trace ( if this doesn't work too well, I'll show you the
 stacktrace by hand below ).
 info locals: displays local variables.
 info args: display current procedure arguments.
 set args: will set argc & argv each time the victim program is invoked.
 set <variable>=value
 set argc=100
 set $pc=0
 Modifying execution
 -------------------
 step: steps n lines of sourcecode
 step steps 1 line.
 step 100 steps 100 lines of code.
 next: like step except this will not step into subroutines
 stepi: steps a single machine code instruction.
 e.g. stepi 100
 nexti: steps a single machine code instruction but will not step into subroutines.
 finish: will run until exit of the current routine
 run: (re)starts a program
 cont: continues a program
 quit: exits gdb.
 breakpoints
 ------------
 break
 sets a breakpoint
 e.g.
 break main
 break *$pc
 break *0x400618
 heres a really useful one for large programs
 rbr
 Set a breakpoint for all functions matching REGEXP
 e.g.
 rbr 390
 will set a breakpoint with all functions with 390 in their name.
 info breakpoints
 lists all breakpoints
 delete: delete breakpoint by number or delete them all
 e.g.
 delete 1 will delete the first breakpoint
 delete will delete them all
 watch: This will set a watchpoint ( usually hardware assisted ),
 This will watch a variable till it changes
 e.g.
 watch cnt, will watch the variable cnt till it changes.
 As an aside unfortunately gdb's, architecture independent watchpoint code
 is inconsistent & not very good, watchpoints usually work but not always.
 info watchpoints: Display currently active watchpoints
 condition: ( another useful one )
 Specify breakpoint number N to break only if COND is true.
 Usage is `condition N COND', where N is an integer and COND is an
 expression to be evaluated whenever breakpoint N is reached.
 User defined functions/macros
 -----------------------------
 define: ( Note this is very very useful,simple & powerful )
 usage define <name> <list of commands> end
 examples which you should consider putting into .gdbinit in your home directory
 define d
 stepi
 disassemble $pc $pc+10
 end
 define e
 nexti
 disassemble $pc $pc+10
 end
 Other hard to classify stuff
 ----------------------------
 signal n:
 sends the victim program a signal.
 e.g. signal 3 will send a SIGQUIT.
 info signals:
 what gdb does when the victim receives certain signals.
 list:
 e.g.
 list lists current function source
 list 1,10 list first 10 lines of current file.
 list test.c:1,10
 directory:
 Adds directories to be searched for source if gdb cannot find the source.
 (note it is a bit sensititive about slashes) 
 e.g. To add the root of the filesystem to the searchpath do
 directory //
 call <function>
 This calls a function in the victim program, this is pretty powerful
 e.g.
 (gdb) call printf("hello world")
 outputs:
 $1 = 11 
 You might now be thinking that the line above didn't work, something extra had to be done.
 (gdb) call fflush(stdout)
 hello world$2 = 0
 As an aside the debugger also calls malloc & free under the hood 
 to make space for the "hello world" string.
 hints
 -----
 1) command completion works just like bash 
 ( if you are a bad typist like me this really helps )
 e.g. hit br <TAB> & cursor up & down :-).
 2) if you have a debugging problem that takes a few steps to recreate
 put the steps into a file called .gdbinit in your current working directory
 if you have defined a few extra useful user defined commands put these in 
 your home directory & they will be read each time gdb is launched.
 A typical .gdbinit file might be.
 break main
 run
 break runtime_exception
 cont 
 stack chaining in gdb by hand
 -----------------------------
 This is done using a the same trick described for VM 
 p/x (*($sp+56))&0x7fffffff get the first backchain.
 For z/Architecture
 Replace 56 with 112 & ignore the &0x7fffffff
 in the macros below & do nasty casts to longs like the following
 as gdb unfortunately deals with printed arguments as ints which
 messes up everything.
 i.e. here is a 3rd backchain dereference
 p/x *(long *)(***(long ***)$sp+112)
 this outputs 
 $5 = 0x528f18 
 on my machine.
 Now you can use 
 info symbol (*($sp+56))&0x7fffffff 
 you might see something like.
 rl_getc + 36 in section .text  telling you what is located at address 0x528f18
 Now do.
 p/x (*(*$sp+56))&0x7fffffff 
 This outputs
 $6 = 0x528ed0
 Now do.
 info symbol (*(*$sp+56))&0x7fffffff
 rl_read_key + 180 in section .text
 now do
 p/x (*(**$sp+56))&0x7fffffff
 & so on.
 Disassembling instructions without debug info
 ---------------------------------------------
 gdb typically complains if there is a lack of debugging
 symbols in the disassemble command with 
 "No function contains specified address." To get around
 this do 
 x/<number lines to disassemble>xi <address>
 e.g.
 x/20xi 0x400730
 Note: Remember gdb has history just like bash you don't need to retype the
 whole line just use the up & down arrows.
 For more info
 -------------
 From your linuxbox do 
 man gdb or info gdb.
 core dumps
 ----------
 What a core dump ?,
 A core dump is a file generated by the kernel ( if allowed ) which contains the registers,
 & all active pages of the program which has crashed.
 From this file gdb will allow you to look at the registers & stack trace & memory of the
 program as if it just crashed on your system, it is usually called core & created in the
 current working directory.
 This is very useful in that a customer can mail a core dump to a technical support department
 & the technical support department can reconstruct what happened.
 Provided the have an identical copy of this program with debugging symbols compiled in & 
 the source base of this build is available.
 In short it is far more useful than something like a crash log could ever hope to be.
 In theory all that is missing to restart a core dumped program is a kernel patch which
 will do the following.
 1) Make a new kernel task structure
 2) Reload all the dumped pages back into the kernel's memory management structures.
 3) Do the required clock fixups
 4) Get all files & network connections for the process back into an identical state ( really difficult ).
 5) A few more difficult things I haven't thought of.
 Why have I never seen one ?.
 Probably because you haven't used the command 
 ulimit -c unlimited in bash
 to allow core dumps, now do 
 ulimit -a 
 to verify that the limit was accepted.
 A sample core dump
 To create this I'm going to do
 ulimit -c unlimited
 gdb 
 to launch gdb (my victim app. ) now be bad & do the following from another 
 telnet/xterm session to the same machine
 ps -aux | grep gdb
 kill -SIGSEGV <gdb's pid>
 or alternatively use killall -SIGSEGV gdb if you have the killall command.
 Now look at the core dump.
-./gdb ./gdb core
+./gdb core
 Displays the following
 GNU gdb 4.18
 Copyright 1998 Free Software Foundation, Inc.
 GDB is free software, covered by the GNU General Public License, and you are
 welcome to change it and/or distribute copies of it under certain conditions.
 Type "show copying" to see the conditions.
 There is absolutely no warranty for GDB.  Type "show warranty" for details.
 This GDB was configured as "s390-ibm-linux"...
 Core was generated by `./gdb'.
 Program terminated with signal 11, Segmentation fault.
 Reading symbols from /usr/lib/libncurses.so.4...done.
 Reading symbols from /lib/libm.so.6...done.
 Reading symbols from /lib/libc.so.6...done.
 Reading symbols from /lib/ld-linux.so.2...done.
 #0  0x40126d1a in read () from /lib/libc.so.6
 Setting up the environment for debugging gdb.
 Breakpoint 1 at 0x4dc6f8: file utils.c, line 471.
 Breakpoint 2 at 0x4d87a4: file top.c, line 2609.
 (top-gdb) info stack
 #0  0x40126d1a in read () from /lib/libc.so.6
 #1  0x528f26 in rl_getc (stream=0x7ffffde8) at input.c:402
 #2  0x528ed0 in rl_read_key () at input.c:381
 #3  0x5167e6 in readline_internal_char () at readline.c:454
 #4  0x5168ee in readline_internal_charloop () at readline.c:507
 #5  0x51692c in readline_internal () at readline.c:521
 #6  0x5164fe in readline (prompt=0x7ffff810 "\177ÿøx\177ÿ÷Ø\177ÿøxÀ")
    at readline.c:349
 #7  0x4d7a8a in command_line_input (prrompt=0x564420 "(gdb) ", repeat=1,
    annotation_suffix=0x4d6b44 "prompt") at top.c:2091
 #8  0x4d6cf0 in command_loop () at top.c:1345
 #9  0x4e25bc in main (argc=1, argv=0x7ffffdf4) at main.c:635
 LDD
 ===
 This is a program which lists the shared libraries which a library needs,
 Note you also get the relocations of the shared library text segments which
 help when using objdump --source.
 e.g.
 ldd ./gdb
 outputs
 libncurses.so.4 => /usr/lib/libncurses.so.4 (0x40018000)
 libm.so.6 => /lib/libm.so.6 (0x4005e000)
 libc.so.6 => /lib/libc.so.6 (0x40084000)
 /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
 Debugging shared libraries
 ==========================
 Most programs use shared libraries, however it can be very painful
 when you single step instruction into a function like printf for the 
 first time & you end up in functions like _dl_runtime_resolve this is
 the ld.so doing lazy binding, lazy binding is a concept in ELF where 
 shared library functions are not loaded into memory unless they are 
 actually used, great for saving memory but a pain to debug.
 To get around this either relink the program -static or exit gdb type 
 export LD_BIND_NOW=true this will stop lazy binding & restart the gdb'ing 
 the program in question.
 Debugging modules
 =================
 As modules are dynamically loaded into the kernel their address can be
 anywhere to get around this use the -m option with insmod to emit a load
 map which can be piped into a file if required.
 The proc file system
 ====================
 What is it ?.
 It is a filesystem created by the kernel with files which are created on demand
 by the kernel if read, or can be used to modify kernel parameters,
 it is a powerful concept.
 e.g.
 cat /proc/sys/net/ipv4/ip_forward 
 On my machine outputs 
 0 
 telling me ip_forwarding is not on to switch it on I can do
 echo 1 >  /proc/sys/net/ipv4/ip_forward
 cat it again
 cat /proc/sys/net/ipv4/ip_forward 
 On my machine now outputs
 1
 IP forwarding is on.
 There is a lot of useful info in here best found by going in & having a look around,
 so I'll take you through some entries I consider important.
 All the processes running on the machine have there own entry defined by
 /proc/<pid>
 So lets have a look at the init process
 cd /proc/1
 cat cmdline
 emits
 init [2]
 cd /proc/1/fd
 This contains numerical entries of all the open files,
 some of these you can cat e.g. stdout (2)
 cat /proc/29/maps
 on my machine emits
 00400000-00478000 r-xp 00000000 5f:00 4103       /bin/bash
 00478000-0047e000 rw-p 00077000 5f:00 4103       /bin/bash
 0047e000-00492000 rwxp 00000000 00:00 0
 40000000-40015000 r-xp 00000000 5f:00 14382      /lib/ld-2.1.2.so
 40015000-40016000 rw-p 00014000 5f:00 14382      /lib/ld-2.1.2.so
 40016000-40017000 rwxp 00000000 00:00 0
 40017000-40018000 rw-p 00000000 00:00 0
 40018000-4001b000 r-xp 00000000 5f:00 14435      /lib/libtermcap.so.2.0.8
 4001b000-4001c000 rw-p 00002000 5f:00 14435      /lib/libtermcap.so.2.0.8
 4001c000-4010d000 r-xp 00000000 5f:00 14387      /lib/libc-2.1.2.so
 4010d000-40111000 rw-p 000f0000 5f:00 14387      /lib/libc-2.1.2.so
 40111000-40114000 rw-p 00000000 00:00 0
 40114000-4011e000 r-xp 00000000 5f:00 14408      /lib/libnss_files-2.1.2.so
 4011e000-4011f000 rw-p 00009000 5f:00 14408      /lib/libnss_files-2.1.2.so
 7fffd000-80000000 rwxp ffffe000 00:00 0
 Showing us the shared libraries init uses where they are in memory
 & memory access permissions for each virtual memory area.
 /proc/1/cwd is a softlink to the current working directory.
 /proc/1/root is the root of the filesystem for this process. 
 /proc/1/mem is the current running processes memory which you
 can read & write to like a file.
 strace uses this sometimes as it is a bit faster than the
 rather inefficient ptrace interface for peeking at DATA.
 cat status 
 Name:   init
 State:  S (sleeping)
 Pid:    1
 PPid:   0
 Uid:    0       0       0       0
 Gid:    0       0       0       0
 Groups:
 VmSize:      408 kB
 VmLck:         0 kB
 VmRSS:       208 kB
 VmData:       24 kB
 VmStk:         8 kB
 VmExe:       368 kB
 VmLib:         0 kB
 SigPnd: 0000000000000000
 SigBlk: 0000000000000000
 SigIgn: 7fffffffd7f0d8fc
 SigCgt: 00000000280b2603
 CapInh: 00000000fffffeff
 CapPrm: 00000000ffffffff
 CapEff: 00000000fffffeff
 User PSW:    070de000 80414146
 task: 004b6000 tss: 004b62d8 ksp: 004b7ca8 pt_regs: 004b7f68
 User GPRS:
 00000400  00000000  0000000b  7ffffa90
 00000000  00000000  00000000  0045d9f4
 0045cafc  7ffffa90  7fffff18  0045cb08
 00010400  804039e8  80403af8  7ffff8b0
 User ACRS:
 00000000  00000000  00000000  00000000
 00000001  00000000  00000000  00000000
 00000000  00000000  00000000  00000000
 00000000  00000000  00000000  00000000
 Kernel BackChain  CallChain    BackChain  CallChain
       004b7ca8   8002bd0c     004b7d18   8002b92c
       004b7db8   8005cd50     004b7e38   8005d12a
       004b7f08   80019114                     
 Showing among other things memory usage & status of some signals &
 the processes'es registers from the kernel task_structure
 as well as a backchain which may be useful if a process crashes
 in the kernel for some unknown reason.
 Some driver debugging techniques
 ================================
 debug feature
 -------------
 Some of our drivers now support a "debug feature" in
 /proc/s390dbf see s390dbf.txt in the linux/Documentation directory
 for more info.
 e.g. 
 to switch on the lcs "debug feature"
 echo 5 > /proc/s390dbf/lcs/level
 & then after the error occurred.
 cat /proc/s390dbf/lcs/sprintf >/logfile
 the logfile now contains some information which may help
 tech support resolve a problem in the field.
 high level debugging network drivers
 ------------------------------------
 ifconfig is a quite useful command
 it gives the current state of network drivers.
 If you suspect your network device driver is dead
 one way to check is type 
 ifconfig <network device> 
 e.g. tr0
 You should see something like
 tr0       Link encap:16/4 Mbps Token Ring (New)  HWaddr 00:04:AC:20:8E:48
          inet addr:9.164.185.132  Bcast:9.164.191.255  Mask:255.255.224.0
          UP BROADCAST RUNNING MULTICAST  MTU:2000  Metric:1
          RX packets:246134 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
 if the device doesn't say up
 try
 /etc/rc.d/init.d/network start 
 ( this starts the network stack & hopefully calls ifconfig tr0 up ).
 ifconfig looks at the output of /proc/net/dev & presents it in a more presentable form
 Now ping the device from a machine in the same subnet.
 if the RX packets count & TX packets counts don't increment you probably
 have problems.
 next 
 cat /proc/net/arp
 Do you see any hardware addresses in the cache if not you may have problems.
 Next try
 ping -c 5 <broadcast_addr> i.e. the Bcast field above in the output of
 ifconfig. Do you see any replies from machines other than the local machine
 if not you may have problems. also if the TX packets count in ifconfig
 hasn't incremented either you have serious problems in your driver 
 (e.g. the txbusy field of the network device being stuck on ) 
 or you may have multiple network devices connected.
 chandev
 -------
 There is a new device layer for channel devices, some
 drivers e.g. lcs are registered with this layer.
 If the device uses the channel device layer you'll be
 able to find what interrupts it uses & the current state 
 of the device.
 See the manpage chandev.8 &type cat /proc/chandev for more info.
 Starting points for debugging scripting languages etc.
 ======================================================
 bash/sh
 bash -x <scriptname>
 e.g. bash -x /usr/bin/bashbug
 displays the following lines as it executes them.
 + MACHINE=i586
 + OS=linux-gnu
 + CC=gcc
 + CFLAGS= -DPROGRAM='bash' -DHOSTTYPE='i586' -DOSTYPE='linux-gnu' -DMACHTYPE='i586-pc-linux-gnu' -DSHELL -DHAVE_CONFIG_H   -I. -I. -I./lib -O2 -pipe
 + RELEASE=2.01
 + PATCHLEVEL=1
 + RELSTATUS=release
 + MACHTYPE=i586-pc-linux-gnu   
 perl -d <scriptname> runs the perlscript in a fully interactive debugger
 <like gdb>.
 Type 'h' in the debugger for help.
 for debugging java type
 jdb <filename> another fully interactive gdb style debugger.
 & type ? in the debugger for help.
 Dumptool & Lcrash ( lkcd )
 ==========================
 Michael Holzheu & others here at IBM have a fairly mature port of 
 SGI's lcrash tool which allows one to look at kernel structures in a
 running kernel.
 It also complements a tool called dumptool which dumps all the kernel's
 memory pages & registers to either a tape or a disk.
 This can be used by tech support or an ambitious end user do
 post mortem debugging of a machine like gdb core dumps.
 Going into how to use this tool in detail will be explained
 in other documentation supplied by IBM with the patches & the 
 lcrash homepage http://oss.sgi.com/projects/lkcd/ & the lcrash manpage.
 How they work
 -------------
 Lcrash is a perfectly normal program,however, it requires 2 
 additional files, Kerntypes which is built using a patch to the 
 linux kernel sources in the linux root directory & the System.map.
-Kerntypes is an an objectfile whose sole purpose in life
+Kerntypes is an objectfile whose sole purpose in life
 is to provide stabs debug info to lcrash, to do this
 Kerntypes is built from kerntypes.c which just includes the most commonly
 referenced header files used when debugging, lcrash can then read the
 .stabs section of this file.
 Debugging a live system it uses /dev/mem
 alternatively for post mortem debugging it uses the data 
 collected by dumptool.
 SysRq
 =====
 This is now supported by linux for s/390 & z/Architecture.
 To enable it do compile the kernel with 
 Kernel Hacking -> Magic SysRq Key Enabled
 echo "1" > /proc/sys/kernel/sysrq
 also type
 echo "8" >/proc/sys/kernel/printk
 To make printk output go to console.
 On 390 all commands are prefixed with
 ^-
 e.g.
 ^-t will show tasks.
 ^-? or some unknown command will display help.
 The sysrq key reading is very picky ( I have to type the keys in an
 xterm session & paste them  into the x3270 console )
 & it may be wise to predefine the keys as described in the VM hints above
 This is particularly useful for syncing disks unmounting & rebooting
 if the machine gets partially hung.
 Read Documentation/sysrq.txt for more info
 References:
 ===========
 Enterprise Systems Architecture Reference Summary
 Enterprise Systems Architecture Principles of Operation
 Hartmut Penners s390 stack frame sheet.
 IBM Mainframe Channel Attachment a technology brief from a CISCO webpage
 Various bits of man & info pages of Linux.
 Linux & GDB source.
 Various info & man pages.
 CMS Help on tracing commands.
 Linux for s/390 Elf Application Binary Interface
 Linux for z/Series Elf Application Binary Interface ( Both Highly Recommended )
 z/Architecture Principles of Operation SA22-7832-00
 Enterprise Systems Architecture/390 Reference Summary SA22-7209-01 & the
 Enterprise Systems Architecture/390 Principles of Operation SA22-7201-05
 Special Thanks
 ==============
 Special thanks to Neale Ferguson who maintains a much
 prettier HTML version of this page at
 http://penguinvm.princeton.edu/notes.html#Debug390
 Bob Grainger Stefan Bader & others for reporting bugs
 S390 Debug Feature
 ==================
 files: arch/s390/kernel/debug.c
       include/asm-s390/debug.h
 Description:
 ------------
 The goal of this feature is to provide a kernel debug logging API 
 where log records can be stored efficiently in memory, where each component 
 (e.g. device drivers) can have one separate debug log.
 One purpose of this is to inspect the debug logs after a production system crash
 in order to analyze the reason for the crash.
 If the system still runs but only a subcomponent which uses dbf fails,
 it is possible to look at the debug logs on a live system via the Linux
 debugfs filesystem.
 The debug feature may also very useful for kernel and driver development.
 Design:
 -------
 Kernel components (e.g. device drivers) can register themselves at the debug 
 feature with the function call debug_register(). This function initializes a 
 debug log for the caller. For each debug log exists a number of debug areas 
 where exactly one is active at one time.  Each debug area consists of contiguous
 pages in memory. In the debug areas there are stored debug entries (log records)
 which are written by event- and exception-calls. 
 An event-call writes the specified debug entry to the active debug
 area and updates the log pointer for the active area. If the end 
 of the active debug area is reached, a wrap around is done (ring buffer) 
 and the next debug entry will be written at the beginning of the active 
 debug area.
 An exception-call writes the specified debug entry to the log and
 switches to the next debug area. This is done in order to be sure
 that the records which describe the origin of the exception are not
 overwritten when a wrap around for the current area occurs.
 The debug areas itselve are also ordered in form of a ring buffer. 
 When an exception is thrown in the last debug area, the following debug 
 entries are then written again in the very first area.
 There are three versions for the event- and exception-calls: One for
 logging raw data, one for text and one for numbers.
 Each debug entry contains the following data:
 - Timestamp
 - Cpu-Number of calling task
 - Level of debug entry (0...6)
 - Return Address to caller
 - Flag, if entry is an exception or not
 The debug logs can be inspected in a live system through entries in
 the debugfs-filesystem. Under the toplevel directory "s390dbf" there is
 a directory for each registered component, which is named like the
 corresponding component. The debugfs normally should be mounted to
 /sys/kernel/debug therefore the debug feature can be accessed unter
 /sys/kernel/debug/s390dbf.
 The content of the directories are files which represent different views
 to the debug log. Each component can decide which views should be
 used through registering them with the function debug_register_view().
 Predefined views for hex/ascii, sprintf and raw binary data are provided.
 It is also possible to define other views. The content of
 a view can be inspected simply by reading the corresponding debugfs file.
-All debug logs have an an actual debug level (range from 0 to 6).
+All debug logs have an actual debug level (range from 0 to 6).
 The default level is 3. Event and Exception functions have a 'level'
 parameter. Only debug entries with a level that is lower or equal
 than the actual level are written to the log. This means, when
 writing events, high priority log entries should have a low level
 value whereas low priority entries should have a high one.
 The actual debug level can be changed with the help of the debugfs-filesystem
 through writing a number string "x" to the 'level' debugfs file which is
 provided for every debug log. Debugging can be switched off completely
 by using "-" on the 'level' debugfs file.
 Example:
 > echo "-" > /sys/kernel/debug/s390dbf/dasd/level
 It is also possible to deactivate the debug feature globally for every
 debug log. You can change the behavior using  2 sysctl parameters in
 /proc/sys/s390dbf:
 There are currently 2 possible triggers, which stop the debug feature
 globally. The first possibility is to use the "debug_active" sysctl. If
 set to 1 the debug feature is running. If "debug_active" is set to 0 the
 debug feature is turned off.
 The second trigger which stops the debug feature is an kernel oops.
 That prevents the debug feature from overwriting debug information that
 happened before the oops. After an oops you can reactivate the debug feature
 by piping 1 to /proc/sys/s390dbf/debug_active. Nevertheless, its not
 suggested to use an oopsed kernel in an production environment.
 If you want to disallow the deactivation of the debug feature, you can use
 the "debug_stoppable" sysctl. If you set "debug_stoppable" to 0 the debug
 feature cannot be stopped. If the debug feature is already stopped, it
 will stay deactivated.
 Kernel Interfaces:
 ------------------
 ----------------------------------------------------------------------------
 debug_info_t *debug_register(char *name, int pages, int nr_areas,
                             int buf_size);
 Parameter:    name:        Name of debug log (e.g. used for debugfs entry)
              pages:       number of pages, which will be allocated per area
              nr_areas:    number of debug areas
              buf_size:    size of data area in each debug entry
 Return Value: Handle for generated debug area   
              NULL if register failed 
 Description:  Allocates memory for a debug log     
              Must not be called within an interrupt handler 
 ---------------------------------------------------------------------------
 void debug_unregister (debug_info_t * id);
 Parameter:     id:   handle for debug log  
 Return Value:  none 
 Description:   frees memory for a debug log     
               Must not be called within an interrupt handler 
 ---------------------------------------------------------------------------
 void debug_set_level (debug_info_t * id, int new_level);
 Parameter:     id:        handle for debug log  
               new_level: new debug level 
 Return Value:  none 
 Description:   Sets new actual debug level if new_level is valid. 
 ---------------------------------------------------------------------------
 void debug_stop_all(void);
 Parameter:     none
 Return Value:  none
 Description:   stops the debug feature if stopping is allowed. Currently
               used in case of a kernel oops.
 ---------------------------------------------------------------------------
 debug_entry_t* debug_event (debug_info_t* id, int level, void* data, 
                            int length);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   pointer to data for debug entry  
               length: length of data in bytes       
 Return Value:  Address of written debug entry 
 Description:   writes debug entry to active debug area (if level <= actual 
               debug level)    
 ---------------------------------------------------------------------------
 debug_entry_t* debug_int_event (debug_info_t * id, int level, 
                                unsigned int data);
 debug_entry_t* debug_long_event(debug_info_t * id, int level,
                                unsigned long data);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   integer value for debug entry           
 Return Value:  Address of written debug entry 
 Description:   writes debug entry to active debug area (if level <= actual 
               debug level)    
 ---------------------------------------------------------------------------
 debug_entry_t* debug_text_event (debug_info_t * id, int level, 
                                 const char* data);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   string for debug entry  
 Return Value:  Address of written debug entry 
 Description:   writes debug entry in ascii format to active debug area 
               (if level <= actual debug level)     
 ---------------------------------------------------------------------------
 debug_entry_t* debug_sprintf_event (debug_info_t * id, int level, 
                                    char* string,...);
 Parameter:     id:    handle for debug log 
               level: debug level
               string: format string for debug entry 
               ...: varargs used as in sprintf()
 Return Value:  Address of written debug entry
 Description:   writes debug entry with format string and varargs (longs) to 
               active debug area (if level $<=$ actual debug level). 
               floats and long long datatypes cannot be used as varargs.
 ---------------------------------------------------------------------------
 debug_entry_t* debug_exception (debug_info_t* id, int level, void* data, 
                                int length);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   pointer to data for debug entry  
               length: length of data in bytes       
 Return Value:  Address of written debug entry 
 Description:   writes debug entry to active debug area (if level <= actual 
               debug level) and switches to next debug area  
 ---------------------------------------------------------------------------
 debug_entry_t* debug_int_exception (debug_info_t * id, int level, 
                                    unsigned int data);
 debug_entry_t* debug_long_exception(debug_info_t * id, int level,
                                    unsigned long data);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   integer value for debug entry           
 Return Value:  Address of written debug entry 
 Description:   writes debug entry to active debug area (if level <= actual 
               debug level) and switches to next debug area  
 ---------------------------------------------------------------------------
 debug_entry_t* debug_text_exception (debug_info_t * id, int level, 
                                     const char* data);
 Parameter:     id:     handle for debug log  
               level:  debug level           
               data:   string for debug entry  
 Return Value:  Address of written debug entry 
 Description:   writes debug entry in ascii format to active debug area 
               (if level <= actual debug level) and switches to next debug 
               area  
 ---------------------------------------------------------------------------
 debug_entry_t* debug_sprintf_exception (debug_info_t * id, int level,
                                        char* string,...);
 Parameter:     id:    handle for debug log  
               level: debug level  
               string: format string for debug entry  
               ...: varargs used as in sprintf()
 Return Value:  Address of written debug entry 
 Description:   writes debug entry with format string and varargs (longs) to 
               active debug area (if level $<=$ actual debug level) and
               switches to next debug area. 
               floats and long long datatypes cannot be used as varargs.
 ---------------------------------------------------------------------------
 int debug_register_view (debug_info_t * id, struct debug_view *view);
 Parameter:     id:    handle for debug log  
               view:  pointer to debug view struct 
 Return Value:  0  : ok 
               < 0: Error 
 Description:   registers new debug view and creates debugfs dir entry
 ---------------------------------------------------------------------------
 int debug_unregister_view (debug_info_t * id, struct debug_view *view); 
 Parameter:     id:    handle for debug log  
               view:  pointer to debug view struct 
 Return Value:  0  : ok 
               < 0: Error 
 Description:   unregisters debug view and removes debugfs dir entry
 Predefined views:
 -----------------
 extern struct debug_view debug_hex_ascii_view;
 extern struct debug_view debug_raw_view;
 extern struct debug_view debug_sprintf_view;
 Examples
 --------
 /*
 * hex_ascii- + raw-view Example
 */
 #include <linux/init.h>
 #include <asm/debug.h>
 static debug_info_t* debug_info;
 static int init(void)
 {
    /* register 4 debug areas with one page each and 4 byte data field */
    debug_info = debug_register ("test", 1, 4, 4 );
    debug_register_view(debug_info,&debug_hex_ascii_view);
    debug_register_view(debug_info,&debug_raw_view);
    debug_text_event(debug_info, 4 , "one ");
    debug_int_exception(debug_info, 4, 4711);
    debug_event(debug_info, 3, &debug_info, 4);
    return 0;
 }
 static void cleanup(void)
 {
    debug_unregister (debug_info);
 }
 module_init(init);
 module_exit(cleanup);
 ---------------------------------------------------------------------------
 /*
 * sprintf-view Example
 */
 #include <linux/init.h>
 #include <asm/debug.h>
 static debug_info_t* debug_info;
 static int init(void)
 {
    /* register 4 debug areas with one page each and data field for */
    /* format string pointer + 2 varargs (= 3 * sizeof(long))       */
    debug_info = debug_register ("test", 1, 4, sizeof(long) * 3);
    debug_register_view(debug_info,&debug_sprintf_view);
    debug_sprintf_event(debug_info, 2 , "first event in %s:%i\n",__FILE__,__LINE__);
    debug_sprintf_exception(debug_info, 1, "pointer to debug info: %p\n",&debug_info);
    return 0;
 }
 static void cleanup(void)
 {
    debug_unregister (debug_info);
 }
 module_init(init);
 module_exit(cleanup);
 Debugfs Interface
 ----------------
 Views to the debug logs can be investigated through reading the corresponding 
 debugfs-files:
 Example:
 > ls /sys/kernel/debug/s390dbf/dasd
 flush  hex_ascii  level pages raw
 > cat /sys/kernel/debug/s390dbf/dasd/hex_ascii | sort +1
 00 00974733272:680099 2 - 02 0006ad7e  07 ea 4a 90 | ....
 00 00974733272:682210 2 - 02 0006ade6  46 52 45 45 | FREE
 00 00974733272:682213 2 - 02 0006adf6  07 ea 4a 90 | ....
 00 00974733272:682281 1 * 02 0006ab08  41 4c 4c 43 | EXCP 
 01 00974733272:682284 2 - 02 0006ab16  45 43 4b 44 | ECKD
 01 00974733272:682287 2 - 02 0006ab28  00 00 00 04 | ....
 01 00974733272:682289 2 - 02 0006ab3e  00 00 00 20 | ... 
 01 00974733272:682297 2 - 02 0006ad7e  07 ea 4a 90 | ....
 01 00974733272:684384 2 - 00 0006ade6  46 52 45 45 | FREE
 01 00974733272:684388 2 - 00 0006adf6  07 ea 4a 90 | ....
 See section about predefined views for explanation of the above output!
 Changing the debug level
 ------------------------
 Example:
 > cat /sys/kernel/debug/s390dbf/dasd/level
 3
 > echo "5" > /sys/kernel/debug/s390dbf/dasd/level
 > cat /sys/kernel/debug/s390dbf/dasd/level
 5
 Flushing debug areas
 --------------------
 Debug areas can be flushed with piping the number of the desired
 area (0...n) to the debugfs file "flush". When using "-" all debug areas
 are flushed.
 Examples:
 1. Flush debug area 0:
 > echo "0" > /sys/kernel/debug/s390dbf/dasd/flush
 2. Flush all debug areas:
 > echo "-" > /sys/kernel/debug/s390dbf/dasd/flush
 Changing the size of debug areas
 ------------------------------------
 It is possible the change the size of debug areas through piping
 the number of pages to the debugfs file "pages". The resize request will
 also flush the debug areas.
 Example:
 Define 4 pages for the debug areas of debug feature "dasd":
 > echo "4" > /sys/kernel/debug/s390dbf/dasd/pages
 Stooping the debug feature
 --------------------------
 Example:
 1. Check if stopping is allowed
 > cat /proc/sys/s390dbf/debug_stoppable
 2. Stop debug feature
 > echo 0 > /proc/sys/s390dbf/debug_active
 lcrash Interface
 ----------------
 It is planned that the dump analysis tool lcrash gets an additional command
 's390dbf' to display all the debug logs. With this tool it will be possible 
 to investigate the debug logs on a live system and with a memory dump after 
 a system crash.
 Investigating raw memory
 ------------------------
 One last possibility to investigate the debug logs at a live
 system and after a system crash is to look at the raw memory
 under VM or at the Service Element.
 It is possible to find the anker of the debug-logs through
 the 'debug_area_first' symbol in the System map. Then one has
 to follow the correct pointers of the data-structures defined
 in debug.h and find the debug-areas in memory.
 Normally modules which use the debug feature will also have
 a global variable with the pointer to the debug-logs. Following
 this pointer it will also be possible to find the debug logs in
 memory.
 For this method it is recommended to use '16 * x + 4' byte (x = 0..n)
 for the length of the data field in debug_register() in
 order to see the debug entries well formatted.
 Predefined Views
 ----------------
 There are three predefined views: hex_ascii, raw and sprintf. 
 The hex_ascii view shows the data field in hex and ascii representation 
 (e.g. '45 43 4b 44 | ECKD'). 
 The raw view returns a bytestream as the debug areas are stored in memory.
 The sprintf view formats the debug entries in the same way as the sprintf
 function would do. The sprintf event/exception functions write to the
 debug entry a pointer to the format string (size = sizeof(long)) 
 and for each vararg a long value. So e.g. for a debug entry with a format 
 string plus two varargs one would need to allocate a (3 * sizeof(long)) 
 byte data area in the debug_register() function.
 NOTE: If using the sprintf view do NOT use other event/exception functions
 than the sprintf-event and -exception functions.
 The format of the hex_ascii and sprintf view is as follows:
 - Number of area
 - Timestamp (formatted as seconds and microseconds since 00:00:00 Coordinated 
  Universal Time (UTC), January 1, 1970)
 - level of debug entry
 - Exception flag (* = Exception)
 - Cpu-Number of calling task
 - Return Address to caller
 - data field
 The format of the raw view is:
 - Header as described in debug.h
 - datafield 
 A typical line of the hex_ascii view will look like the following (first line 
 is only for explanation and will not be displayed when 'cating' the view):
 area  time           level exception cpu caller    data (hex + ascii)
 --------------------------------------------------------------------------
 00    00964419409:440690 1 -         00  88023fe   
 Defining views
 --------------
 Views are specified with the 'debug_view' structure. There are defined
 callback functions which are used for reading and writing the debugfs files:
 struct debug_view {
        char name[DEBUG_MAX_PROCF_LEN];  
        debug_prolog_proc_t* prolog_proc; 
        debug_header_proc_t* header_proc;
        debug_format_proc_t* format_proc;
        debug_input_proc_t*  input_proc;
 	void*                private_data;
 };
 where
 typedef int (debug_header_proc_t) (debug_info_t* id,
                                   struct debug_view* view,
                                   int area,
                                   debug_entry_t* entry,
                                   char* out_buf);
 typedef int (debug_format_proc_t) (debug_info_t* id,
                                   struct debug_view* view, char* out_buf,
                                   const char* in_buf);
 typedef int (debug_prolog_proc_t) (debug_info_t* id,
                                   struct debug_view* view,
                                   char* out_buf);
 typedef int (debug_input_proc_t) (debug_info_t* id,
                                  struct debug_view* view,
                                  struct file* file, const char* user_buf,
                                  size_t in_buf_size, loff_t* offset);
 The "private_data" member can be used as pointer to view specific data.
 It is not used by the debug feature itself.
 The output when reading a debugfs file is structured like this:
 "prolog_proc output"
 "header_proc output 1"  "format_proc output 1"
 "header_proc output 2"  "format_proc output 2"
 "header_proc output 3"  "format_proc output 3"
 ...
 When a view is read from the debugfs, the Debug Feature calls the
 'prolog_proc' once for writing the prolog.
 Then 'header_proc' and 'format_proc' are called for each 
 existing debug entry.
 The input_proc can be used to implement functionality when it is written to 
 the view (e.g. like with 'echo "0" > /sys/kernel/debug/s390dbf/dasd/level).
 For header_proc there can be used the default function
-debug_dflt_header_fn() which is defined in in debug.h.
+debug_dflt_header_fn() which is defined in debug.h.
 and which produces the same header output as the predefined views.
 E.g:
 00 00964419409:440761 2 - 00 88023ec
 In order to see how to use the callback functions check the implementation
 of the default views!
 Example
 #include <asm/debug.h>
 #define UNKNOWNSTR "data: %08x"
 const char* messages[] =
 {"This error...........\n",
 "That error...........\n",
 "Problem..............\n",
 "Something went wrong.\n",
 "Everything ok........\n",
 NULL
 };
 static int debug_test_format_fn(
   debug_info_t * id, struct debug_view *view, 
   char *out_buf, const char *in_buf
 )
 {
  int i, rc = 0;
  if(id->buf_size >= 4) {
     int msg_nr = *((int*)in_buf);
     if(msg_nr < sizeof(messages)/sizeof(char*) - 1)
        rc += sprintf(out_buf, "%s", messages[msg_nr]);	
     else
        rc += sprintf(out_buf, UNKNOWNSTR, msg_nr);
  }
 out:
   return rc;
 }
 struct debug_view debug_test_view = {
  "myview",                 /* name of view */
  NULL,                     /* no prolog */
  &debug_dflt_header_fn,    /* default header for each entry */
  &debug_test_format_fn,    /* our own format function */
  NULL,                     /* no input function */
  NULL                      /* no private data */
 };
 =====
 test:
 =====
 debug_info_t *debug_info;
 ...
 debug_info = debug_register ("test", 0, 4, 4 ));
 debug_register_view(debug_info, &debug_test_view);
 for(i = 0; i < 10; i ++) debug_int_event(debug_info, 1, i);
 > cat /sys/kernel/debug/s390dbf/test/myview
 00 00964419734:611402 1 - 00 88042ca   This error...........
 00 00964419734:611405 1 - 00 88042ca   That error...........
 00 00964419734:611408 1 - 00 88042ca   Problem..............
 00 00964419734:611411 1 - 00 88042ca   Something went wrong.
 00 00964419734:611414 1 - 00 88042ca   Everything ok........
 00 00964419734:611417 1 - 00 88042ca   data: 00000005
 00 00964419734:611419 1 - 00 88042ca   data: 00000006
 00 00964419734:611422 1 - 00 88042ca   data: 00000007
 00 00964419734:611425 1 - 00 88042ca   data: 00000008
 00 00964419734:611428 1 - 00 88042ca   data: 00000009
 Sat Jan 18 15:51:45 1997  Richard Henderson  <rth@tamu.edu>
 	* Don't play with usage_count directly, instead hand around
 	the module header and use the module macros.
 Fri May 17 00:00:00 1996  Leonard N. Zubkoff <lnz@dandelion.com>
 	* BusLogic Driver Version 2.0.3 Released.
 Tue Apr 16 21:00:00 1996  Leonard N. Zubkoff <lnz@dandelion.com>
 	* BusLogic Driver Version 1.3.2 Released.
 Sun Dec 31 23:26:00 1995  Leonard N. Zubkoff <lnz@dandelion.com>
 	* BusLogic Driver Version 1.3.1 Released.
 Fri Nov 10 15:29:49 1995  Leonard N. Zubkoff <lnz@dandelion.com>
 	* Released new BusLogic driver.
 Wed Aug  9 22:37:04 1995  Andries Brouwer  <aeb@cwi.nl>
 	As a preparation for new device code, separated the various
 	functions the request->dev field had into the device proper,
 	request->rq_dev and a status field request->rq_status.
 	The 2nd argument of bios_param is now a kdev_t.
 Wed Jul 19 10:43:15 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
        * scsi.c (scsi_proc_info): /proc/scsi/scsi now also lists all
 	attached devices.
 	* scsi_proc.c (proc_print_scsidevice): Added. Used by scsi.c and
 	eata_dma_proc.c to produce some device info for /proc/scsi.
 	* eata_dma.c (eata_queue)(eata_int_handler)(eata_scsi_done):
 	Changed handling of internal SCSI commands send to the HBA.
 Wed Jul 19 10:09:17 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.11 released.
 	* eata_dma.c (eata_queue)(eata_int_handler): Added code to do
 	command latency measurements if requested by root through
 	/proc/scsi interface.
 	Throughout Use HZ constant for time references.
 	* eata_pio.c: Use HZ constant for time references.
 	* aic7xxx.c, aic7xxx.h, aic7xxx_asm.c: Changed copyright from BSD
 	to GNU style.
 	* scsi.h: Added READ_12 command opcode constant
 Wed Jul 19 09:25:30 1995  Michael Neuffer <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.10 released.
 	* scsi_proc.c (dispatch_scsi_info): Removed unused variable.
 Wed Jul 19 09:25:30 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.9 released.
 	* scsi.c Blacklist concept expanded to 'support' more device
 	deficiencies. blacklist[] renamed to device_list[]
 	(scan_scsis): Code cleanup.
 	* scsi_debug.c (scsi_debug_proc_info): Added support to control
 	device lockup simulation via /proc/scsi interface.
 Wed Jul 19 09:22:34 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.7 released.
 	* scsi_proc.c: Fixed a number of bugs in directory handling
 Wed Jul 19 09:18:28 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.5 released.
 	* Native wide, multichannel and /proc/scsi support now in official
 	kernel distribution.
        * scsi.c/h, hosts.c/h et al reindented to increase readability
 	(especially on 80 column wide terminals).
 	* scsi.c, scsi_proc.c, ../../fs/proc/inode.c: Added
 	/proc/scsi/scsi which allows root to scan for hotplugged devices.
 	* scsi.c (scsi_proc_info): Added, to support /proc/scsi/scsi.
 	(scan_scsis): Added some 'spaghetti' code to allow scanning for
 	single devices.
 Thu Jun 20 15:20:27 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
        * proc.c: Renamed to scsi_proc.c
 Mon Jun 12 20:32:45 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* Linux 1.3.0 released.
 Mon May 15 19:33:14 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* scsi.c: Added native multichannel and wide scsi support.
 	* proc.c (dispatch_scsi_info) (build_proc_dir_hba_entries):
 	Updated /proc/scsi interface.
 Thu May  4 17:58:48 1995  Michael Neuffer  <neuffer@goofy.zdv.uni-mainz.de>
 	* sd.c (requeue_sd_request): Zero out the scatterlist only if
 	scsi_malloc returned memory for it.
 	* eata_dma.c (register_HBA) (eata_queue): Add support for
 	large scatter/gather tables and set use_clustering accordingly
 	* hosts.c: Make use_clustering changeable in the Scsi_Host structure.
 Wed Apr 12 15:25:52 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.2.5 released.
 	* buslogic.c: Update to version 1.15 (From Leonard N. Zubkoff).
 	Fixed interrupt routine to avoid races when handling multiple
 	complete commands per interrupt.  Seems to come up with faster
 	cards.
 	* eata_dma.c: Update to 2.3.5r. Modularize. Improved error handling
        throughout and fixed bug interrupt routine which resulted in shifted
        status bytes. Added blink LED state checks for ISA and EISA HBAs.
        Memory management bug seems to have disappeared ==> increasing
        C_P_L_CURRENT_MAX to 16 for now. Decreasing C_P_L_DIV to 3 for
        performance reasons.
 	* scsi.c: If we get a FMK, EOM, or ILI when attempting to scan
 	the bus, assume that it was just noise on the bus, and ignore
 	the device.
 	* scsi.h: Update and add a bunch of missing commands which we
 	were never using.
 	* sd.c: Use restore_flags in do_sd_request - this may result in
 	latency conditions, but it gets rid of races and crashes.
 	Do not save flags again when searching for a second command to
 	queue.
 	* st.c: Use bytes, not STP->buffer->buffer_size when reading
 	from tape.
 Tue Apr  4 09:42:08 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.2.4 released.
 	* st.c: Fix typo - restoring wrong flags.
 Wed Mar 29 06:55:12 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.2.3 released.
 	* st.c: Perform some waiting operations with interrupts off.
 	Is this correct???
 Wed Mar 22 10:34:26 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.2.2 released.
 	* aha152x.c: Modularize.  Add support for PCMCIA.
 	* eata.c: Update to version 2.0.  Fixed bug preventing media
 	detection.  If scsi_register_host returns NULL, fail gracefully.
 	* scsi.c: Detect as NEC (for photo-cd purposes) for the 84
 	and 25 models as "NEC_OLDCDR".
 	* scsi.h: Add define for NEC_OLDCDR
 	* sr.c: Add handling for NEC_OLDCDR.  Treat as unknown.
 	* u14-34f.c: Update to version 2.0.  Fixed same bug as in
 	eata.c.
 Mon Mar  6 11:11:20 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.2.0 released.  Yeah!!!
 	* Minor spelling/punctuation changes throughout.  Nothing
 	substantive.
 Mon Feb 20 21:33:03 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.95 released.
 	* qlogic.c: Update to version 0.41.
 	* seagate.c: Change some message to be more descriptive about what
 	we detected.
 	* sr.c: spelling/whitespace changes.
 Mon Feb 20 21:33:03 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.94 released.
 Mon Feb 20 08:57:17 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.93 released.
 	* hosts.h: Change io_port to long int from short.
 	* 53c7,8xx.c: crash on AEN fixed, SCSI reset is no longer a NOP,
 	  NULL pointer panic on odd UDCs fixed, two bugs in diagnostic output
 	  fixed, should initialize correctly if left running, now loadable,
  	  new memory allocation, extraneous diagnostic output suppressed,
 	  splx() replaced with save/restore flags. [ Drew ]
 	* hosts.c, hosts.h, scsi_ioctl.c, sd.c, sd_ioctl.c, sg.c, sr.c,
 	sr_ioctl.c: Add special junk at end that Emacs will use for
 	formatting the file.
 	* qlogic.c: Update to v0.40a.  Improve parity handling.
 	* scsi.c: Add Hitachi DK312C to blacklist.  Change "};" to "}" in
 	many places.  Use scsi_init_malloc to get command block - may
 	need this to be dma compatible for some host adapters.
 	Restore interrupts after unregistering a host.
 	* sd.c: Use sti instead of restore flags - causes latency problems.
 	* seagate.c: Use controller_type to determine string used when
 	registering irq.
 	* sr.c: More photo-cd hacks to make sure we get the xa stuff right.
 	* sr.h, sr.c: Change is_xa to xa_flags field.
 	* st.c: Disable retries for write operations.
 Wed Feb 15 10:52:56 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.92 released.
 	* eata.c: Update to 1.17.
 	* eata_dma.c: Update to 2.31a. Add more support for /proc/scsi.
        Continuing modularization. Less crashes because of the bug in the
        memory management ==> increase C_P_L_CURRENT_MAX to 10
        and decrease C_P_L_DIV to 4.
 	* hosts.c: If we remove last host registered, reuse host number.
 	When freeing memory from host being deregistered, free extra_bytes
 	too.
 	* scsi.c (scan_scsis): memset(SDpnt, 0) and set SCmd.device to SDpnt.
 	Change memory allocation to work around bugs in __get_dma_pages.
 	Do not free host if usage count is not zero (for modules).
 	* sr_ioctl.c: Increase IOCTL_TIMEOUT to 3000.
 	* st.c: Allow for ST_EXTRA_DEVS in st data structures.
 	* u14-34f.c: Update to 1.17.
 Thu Feb  9 10:11:16 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.91 released.
 	* eata.c: Update to 1.16.  Use wish_block instead of host->block.
 	* hosts.c: Initialize wish_block to 0.
 	* hosts.h: Add wish_block.
 	* scsi.c: Use wish_block as indicator that the host should be added
 	to block list.
 	* sg.c: Add SG_EXTRA_DEVS to number of slots.
 	* u14-34f.c: Use wish_block.
 Tue Feb  7 11:46:04 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.90 released.
 	* eata.c: Change naming from eata_* to eata2x_*.  Now at vers 1.15.
 	Update interrupt handler to take pt_regs as arg.  Allow blocking
 	even if loaded as module.  Initialize target_time_out array.
 	Do not put sti(); in timing loop.
 	* hosts.c: Do not reuse host numbers.
 	Use scsi_make_blocked_list to generate blocking list.
 	* script_asm.pl:  Beats me.  Don't know perl.  Something to do with
 	phase index.
 	* scsi.c (scsi_make_blocked_list): New function - code copied from
 	hosts.c.
 	* scsi.c: Update code to disable photo CD for Toshiba cdroms.
 	Use just manufacturer name, not model number.
 	* sr.c: Fix setting density for Toshiba drives.
 	* u14-34f.c: Clear target_time_out array during reset.
 Wed Feb  1 09:20:45 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.89 released.
 	* Makefile, u14-34f.c: Modularize.
 	* Makefile, eata.c: Modularize.  Now version 1.14
 	* NCR5380.c: Update interrupt handler with new arglist.  Minor
 	cleanups.
 	* eata_dma.c: Begin to modularize.  Add hooks for /proc/scsi.
 	New version 2.3.0a. Add code in interrupt handler to allow
        certain CDROM drivers to be detected which return a
        CHECK_CONDITION during SCSI bus scan. Add opcode check to get
        all DATA IN and DATA OUT phases right. Utilize HBA_interpret flag.
 	Improvements in HBA identification. Various other minor stuff.
 	* hosts.c: Initialize ->dma_channel and ->io_port when registering
 	a new host.
 	* qlogic.c: Modularize and add PCMCIA support.
 	* scsi.c: Add Hitachi to blacklist.
 	* scsi.c: Change default to no lun scan (too many problem devices).
 	* scsi.h: Define QUEUE_FULL condition.
 	* sd.c: Do not check for non-existent partition until after
 	new media check.
 	* sg.c: Undo previous change which was wrong.
 	* sr_ioctl.c: Increase IOCTL_TIMEOUT to 2000.
 	* st.c: Patches from Kai - improve filemark handling.
 Tue Jan 31 17:32:12 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.88 released.
 	* Throughout - spelling/grammar fixups.
 	* scsi.c: Make sure that all buffers are 16 byte aligned - some
 	drivers (buslogic) need this.
 	* scsi.c (scan_scsis): Remove message printed.
 	* scsi.c (scsi_init): Move message here.
 Mon Jan 30 06:40:25 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.87 released.
 	* sr.c: Photo-cd related changes. (Gerd Knorr??).
 	* st.c: Changes from Kai related to EOM detection.
 Mon Jan 23 23:53:10 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.86 released.
 	* 53c7,8xx.h: Change SG size to 127.
 	* eata_dma: Update to version 2.10i. Remove bug in the registration
        of multiple HBAs and channels. Minor other improvements and stylistic
        changes.
 	* scsi.c: Test for Toshiba XM-3401TA and exclude from detection
 	as toshiba drive - photo cd does not work with this drive.
 	* sr.c:  Update photocd code.
 Mon Jan 23 23:53:10 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.85 released.
 	* st.c, st_ioctl.c, sg.c, sd_ioctl.c, scsi_ioctl.c, hosts.c:
 	include linux/mm.h
 	* qlogic.c, buslogic.c, aha1542.c: Include linux/module.h.
 Sun Jan 22 22:08:46 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.84 released.
 	* Makefile: Support for loadable QLOGIC boards.
 	* aha152x.c: Update to version 1.8 from Juergen.
 	* eata_dma.c: Update from Michael Neuffer.
        Remove hard limit of 2 commands per lun and make it better
        configurable. Improvements in HBA identification.
 	* in2000.c: Fix biosparam to support large disks.
 	* qlogic.c: Minor changes (change sti -> restore_flags).
 Wed Jan 18 23:33:09 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.83 released.
 	* aha1542.c(aha1542_intr_handle): Use arguments handed down to find
 	which irq.
 	* buslogic.c: Likewise.
 	* eata_dma.c: Use min of 2 cmd_per_lun for OCS_enabled boards.
 	* scsi.c: Make RECOVERED_ERROR a SUGGEST_IS_OK.
 	* sd.c: Fail if we are opening a non-existent partition.
 	* sr.c: Bump SR_TIMEOUT to 15000.
 	Do not probe for media size at boot time(hard on changers).
 	Flag device as needing sector size instead.
 	* sr_ioctl.c: Remove CDROMMULTISESSION_SYS ioctl.
 	* ultrastor.c: Fix bug in call to ultrastor_interrupt (wrong #args).
 Mon Jan 16 07:18:23 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.82 released.
 	Throughout.
 	- Change all interrupt handlers to accept new calling convention.
 	In particular, we now receive the irq number as one of the arguments.
 	* More minor spelling corrections in some of the new files.
 	* aha1542.c, buslogic.c: Clean up interrupt handler a little now
 	that we receive the irq as an arg.
 	* aha274x.c: s/snarf_region/request_region/
 	* eata.c: Update to version 1.12.   Fix some comments and display a
 	message if we cannot reserve the port addresses.
 	* u14-34f.c: Update to version 1.13.   Fix some comments and display a
 	message if we cannot reserve the port addresses.
 	* eata_dma.c: Define get_board_data function (send INQUIRY command).
 	Use to improve detection of variants of different DPT boards.  Change
 	version subnumber to "0g".
 	* fdomain.c:  Update to version 5.26.  Improve detection of some boards
 	repackaged by IBM.
 	* scsi.c (scsi_register_host): Change "name" to const char *.
 	* sr.c: Fix problem in set mode command for Toshiba drives.
 	* sr.c: Fix typo from patch 81.
 Fri Jan 13 12:54:46 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.81 released.  Codefreeze for 1.2 release announced.
 	Big changes here.
 	* eata_dma.*: New files from Michael Neuffer.
 	(neuffer@goofy.zdv.uni-mainz.de).  Should support
 	all eata/dpt cards.
 	* hosts.c, Makefile: Add eata_dma.
 	* README.st: Document MTEOM.
 	Patches from me (ERY) to finish support for low-level loadable scsi.
 	It now works, and is actually useful.
 	* Throughout - add new argument to scsi_init_malloc that takes an
 	additional parameter.  This is used as a priority to kmalloc,
 	and you can specify the GFP_DMA flag if you need DMA-able memory.
 	* Makefile: For source files that are loadable, always add name
 	to SCSI_SRCS.  Fill in modules: target.
 	* hosts.c:  Change next_host to next_scsi_host, and make global.
 	Print hosts after we have identified all of them.  Use info()
 	function if present, otherwise use name field.
 	* hosts.h: Change attach function to return int, not void.
 	Define number of device slots to allow for loadable devices.
 	Define tags to tell scsi module code what type of module we
 	are loading.
 	* scsi.c: Fix scan_scsis so that it can be run by a user process.
 	Do not use waiting loops - use up and down mechanism as long
 	as current != task[0].
 	* scsi.c(scan_scsis): Do not use stack variables for I/O - this
 	could be > 16Mb if we are loading a module at runtime (i.e. use
 	scsi_init_malloc to get some memory we know will be safe).
 	* scsi.c: Change dma freelist to be a set of pages.  This allows
 	us to dynamically adjust the size of the list by adding more pages
 	to the pagelist.  Fix scsi_malloc and scsi_free accordingly.
 	* scsi_module.c: Fix include.
 	* sd.c: Declare detach function.  Increment/decrement module usage
 	count as required.  Fix init functions to allow loaded devices.
 	Revalidate all new disks so we get the partition tables.  Define
 	detach function.
 	* sr.c: Likewise.
 	* sg.c: Declare detach function.  Allow attachment of devices on
 	loaded drivers.
 	* st.c: Declare detach function.  Increment/decrement module usage
 	count as required.
 Tue Jan 10 10:09:58 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.79 released.
 	Patch from some undetermined individual who needs to get a life :-).
 	* sr.c: Attacked by spelling bee...
 	Patches from Gerd Knorr:
 	* sr.c: make printk messages for photoCD a little more informative.
 	* sr_ioctl.c: Fix CDROMMULTISESSION_SYS ioctl.
 Mon Jan  9 10:01:37 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.78 released.
 	* Makefile: Add empty modules: target.
 	* Wheee.  Now change register_iomem to request_region.
 	* in2000.c: Bugfix - apparently this is the fix that we have
 	all been waiting for.  It fixes a problem whereby the driver
 	is not stable under heavy load.  Race condition and all that.
 	Patch from Peter Lu.
 Wed Jan  4 21:17:40 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.77 released.
 	* 53c7,8xx.c: Fix from Linus - emulate splx.
 	Throughout:
 		Change "snarf_region" with "register_iomem".
 	* scsi_module.c: New file.  Contains support for low-level loadable
 	  scsi drivers. [ERY].
 	* sd.c: More s/int/long/ changes.
 	* seagate.c: Explicitly include linux/config.h
 	* sg.c: Increment/decrement module usage count on open/close.
 	* sg.c: Be a bit more careful about the user not supplying enough
 	  information for a valid command.  Pass correct size down to
 	  scsi_do_cmd.
 	* sr.c:  More changes for Photo-CD.  This apparently breaks NEC drives.
 	* sr_ioctl.c:  Support CDROMMULTISESSION ioctl.
 Sun Jan  1 19:55:21 1995  Eric Youngdale  (eric@andante)
 	* Linux 1.1.76 released.
 	* constants.c: Add type cast in switch statement.
 	* scsi.c (scsi_free): Change datatype of "offset" to long.
 	  (scsi_malloc): Change a few more variables to long.  Who
 	  did this and why was it important?  64 bit machines?
 	Lots of changes to use save_state/restore_state instead of cli/sti.
 	Files changed include:
 	* aha1542.c:
 	* aha1740.c:
 	* buslogic.c:
 	* in2000.c:
 	* scsi.c:
 	* scsi_debug.c:
 	* sd.c:
 	* sr.c:
 	* st.c:
 Wed Dec 28 16:38:29 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.75 released.
 	* buslogic.c: Spelling fix.
 	* scsi.c: Add HP C1790A and C2500A scanjet to blacklist.
 	* scsi.c: Spelling fixup.
 	* sd.c: Add support for sd_hardsizes (hard sector sizes).
 	* ultrastor.c: Use save_flags/restore_flags instead of cli/sti.
 Fri Dec 23 13:36:25 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.74 released.
 	* README.st: Update from Kai Makisara.
 	* eata.c: New version from Dario - version 1.11.
 	  use scsicam bios_param routine.  Add support for 2011
 	  and 2021 boards.
 	* hosts.c: Add support for blocking.  Linked list automatically
 	  generated when shpnt->block is set.
 	* scsi.c: Add sankyo & HP scanjet to blacklist.  Add support for
 	  kicking things loose when we deadlock.
 	* scsi.c: Recognize scanners and processors in scan_scsis.
 	* scsi_ioctl.h: Increase timeout to 9 seconds.
 	* st.c: New version from Kai - add better support for backspace.
 	* u14-34f.c: New version from Dario.  Supports blocking.
 Wed Dec 14 14:46:30 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.73 released.
 	* buslogic.c: Update from Dave Gentzel.  Version 1.14.
 	  Add module related stuff.   More fault tolerant if out of
 	  DMA memory.
 	* fdomain.c: New version from Rik Faith - version 5.22.  Add support
 	  for ISA-200S SCSI adapter.
 	* hosts.c: Spelling.
 	* qlogic.c: Update to version 0.38a.  Add more support for PCMCIA.
 	* scsi.c: Mask device type with 0x1f during scan_scsis.
 	  Add support for deadlocking, err, make that getting out of
 	  deadlock situations that are created when we allow the user
 	  to limit requests to one host adapter at a time.
 	* scsi.c: Bugfix - pass pid, not SCpnt as second arg to
 	  scsi_times_out.
 	* scsi.c: Restore interrupt state to previous value instead of using
 	  cli/sti pairs.
 	* scsi.c: Add a bunch of module stuff (all commented out for now).
 	* scsi.c: Clean up scsi_dump_status.
 Tue Dec  6 12:34:20 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.72 released.
 	* sg.c: Bugfix - always use sg_free, since we might have big buff.
 Fri Dec  2 11:24:53 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.71 released.
 	* sg.c: Clear buff field when not in use.  Only call scsi_free if
 	non-null.
 	* scsi.h: Call wake_up(&wait_for_request) when done with a
 	command.
 	* scsi.c (scsi_times_out): Pass pid down so that we can protect
 	against race conditions.
 	* scsi.c (scsi_abort): Zero timeout field if we get the
 	NOT_RUNNING message back from low-level driver.
 	* scsi.c (scsi_done): Restore cmd_len, use_sg here.
 	* scsi.c (request_sense): Not here.
 	* hosts.h: Add new forbidden_addr, forbidden_size fields.  Who
 	added these and why????
 	* hosts.c (scsi_mem_init): Mark pages as reserved if they fall in
 	the forbidden regions.  I am not sure - I think this is so that
 	we can deal with boards that do incomplete decoding of their
 	address lines for the bios chips, but I am not entirely sure.
 	* buslogic.c: Set forbidden_addr stuff if using a buggy board.
 	* aha1740.c: Test for NULL pointer in SCtmp.  This should not
 	occur, but a nice message is better than a kernel segfault.
 	* 53c7,8xx.c: Add new PCI chip ID for 815.
 Fri Dec  2 11:24:53 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.70 released.
 	* ChangeLog, st.c: Spelling.
 Tue Nov 29 18:48:42 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.69 released.
 	* u14-34f.h: Non-functional change.  [Dario].
 	* u14-34f.c: Use block field in Scsi_Host to prevent commands from
 	being queued to more than one host at the same time (used when
 	motherboard does not deal with multiple bus-masters very well).
 	Only when SINGLE_HOST_OPERATIONS is defined.
 	Use new cmd_per_lun field.  [Dario]
 	* eata.c: Likewise.
 	* st.c: More changes from Kai.  Add ready flag to indicate drive
 	status.
 	* README.st: Document this.
 	* sr.c: Bugfix (do not subtract CD_BLOCK_OFFSET) for photo-cd
 	code.
 	* sg.c: Bugfix - fix problem where opcode is not correctly set up.
 	* seagate.[c,h]: Use #defines to set driver name.
 	* scsi_ioctl.c: Zero buffer before executing command.
 	* scsi.c: Use new cmd_per_lun field in Scsi_Hosts as appropriate.
 	Add Sony CDU55S to blacklist.
 	* hosts.h: Add new cmd_per_lun field to Scsi_Hosts.
 	* hosts.c: Initialize cmd_per_lun in Scsi_Hosts from template.
 	* buslogic.c: Use cmd_per_lun field - initialize to different
 	values depending upon bus type (i.e. use 1 if ISA, so we do not
 	hog memory).  Use other patches which got lost from 1.1.68.
 	* aha1542.c: Spelling.
 Tue Nov 29 15:43:50 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.68 released.
 	Add support for 12 byte vendor specific commands in scsi-generics,
 	more (i.e. the last mandatory) low-level changes to support
 	loadable modules, plus a few other changes people have requested
 	lately.  Changes by me (ERY) unless otherwise noted.  Spelling
 	changes appear from some unknown corner of the universe.
 	* Throughout: Change COMMAND_SIZE() to use SCpnt->cmd_len.
 	* Throughout: Change info() low level function to take a Scsi_Host
 	pointer.  This way the info function can return specific
 	information about the host in question, if desired.
 	* All low-level drivers: Add NULL in initializer for the
 	usage_count field added to Scsi_Host_Template.
 	* aha152x.[c,h]: Remove redundant info() function.
 	* aha1542.[c,h]: Likewise.
 	* aha1740.[c,h]: Likewise.
 	* aha274x.[c,h]: Likewise.
 	* eata.[c,h]: Likewise.
 	* pas16.[c,h]: Likewise.
 	* scsi_debug.[c,h]: Likewise.
 	* t128.[c,h]: Likewise.
 	* u14-34f.[c,h]: Likewise.
 	* ultrastor.[c,h]: Likewise.
 	* wd7000.[c,h]: Likewise.
 	* aha1542.c: Add support for command line options with lilo to set
 	DMA parameters, I/O port.  From Matt Aarnio.
 	* buslogic.[c,h]: New version (1.13) from Dave Gentzel.
 	* hosts.h: Add new field to Scsi_Hosts "block" to allow blocking
 	all I/O to certain other cards.  Helps prevent problems with some
 	ISA motherboards.
 	* hosts.h: Add usage_count to Scsi_Host_Template.
 	* hosts.h: Add n_io_port to Scsi_Host (used when releasing module).
 	* hosts.c: Initialize block field.
 	* in2000.c: Remove "static" declarations from exported functions.
 	* in2000.h: Likewise.
 	* scsi.c: Correctly set cmd_len field as required.  Save and
 	change setting when doing a request_sense, restore when done.
 	Move abort timeout message.  Fix panic in request_queueable to
 	print correct function name.
 	* scsi.c: When incrementing usage count, walk block linked list
 	for host, and or in SCSI_HOST_BLOCK bit.  When decrementing usage
 	count to 0, clear this bit to allow usage to continue, wake up
 	processes waiting.
 	* scsi_ioctl.c: If we have an info() function, call it, otherwise
 	if we have a "name" field, use it, else do nothing.
 	* sd.c, sr.c: Clear cmd_len field prior to each command we
 	generate.
 	* sd.h: Add "has_part_table" bit to rscsi_disks.
 	* sg.[c,h]: Add support for vendor specific 12 byte commands (i.e.
 	override command length in COMMAND_SIZE).
 	* sr.c: Bugfix from Gerd in photocd code.
 	* sr.c: Bugfix in get_sectorsize - always use scsi_malloc buffer -
 	we cannot guarantee that the stack is < 16Mb.
 Tue Nov 22 15:40:46 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.67 released.
 	* sr.c: Change spelling of manufactor to manufacturer.
 	* scsi.h: Likewise.
 	* scsi.c: Likewise.
 	* qlogic.c: Spelling corrections.
 	* in2000.h: Spelling corrections.
 	* in2000.c: Update from Bill Earnest, change from
 	jshiffle@netcom.com.  Support new bios versions.
 	* README.qlogic: Spelling correction.
 Tue Nov 22 15:40:46 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.66 released.
 	* u14-34f.c: Spelling corrections.
 	* sr.[h,c]: Add support for multi-session CDs from Gerd Knorr.
 	* scsi.h: Add manufactor field for keeping track of device
 	manufacturer.
 	* scsi.c: More spelling corrections.
 	* qlogic.h, qlogic.c, README.qlogic: New driver from Tom Zerucha.
 	* in2000.c, in2000.h: New driver from Brad McLean/Bill Earnest.
 	* fdomain.c: Spelling correction.
 	* eata.c: Spelling correction.
 Fri Nov 18 15:22:44 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.65 released.
 	* eata.h: Update version string to 1.08.00.
 	* eata.c: Set sg_tablesize correctly for DPT PM2012 boards.
 	* aha274x.seq: Spell checking.
 	* README.st: Likewise.
 	* README.aha274x: Likewise.
 	* ChangeLog: Likewise.
 Tue Nov 15 15:35:08 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.64 released.
 	* u14-34f.h: Update version number to 1.10.01.
 	* u14-34f.c: Use Scsi_Host can_queue variable instead of one from template.
 	* eata.[c,h]: New driver for DPT boards from Dario Ballabio.
 	* buslogic.c: Use can_queue field.
 Wed Nov 30 12:09:09 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.63 released.
 	* sd.c: Give I/O error if we attempt 512 byte I/O to a disk with
 	1024 byte sectors.
 	* scsicam.c: Make sure we do read from whole disk (mask off
 	partition).
 	* scsi.c: Use can_queue in Scsi_Host structure.
 	Fix panic message about invalid host.
 	* hosts.c: Initialize can_queue from template.
 	* hosts.h: Add can_queue to Scsi_Host structure.
 	* aha1740.c: Print out warning about NULL ecbptr.
 Fri Nov  4 12:40:30 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.62 released.
 	* fdomain.c: Update to version 5.20. (From Rik Faith).  Support
 	BIOS version 3.5.
 	* st.h: Add ST_EOD symbol.
 	* st.c: Patches from Kai Makisara - support additional densities,
 	add support for MTFSS, MTBSS, MTWSM commands.
 	* README.st: Update to document new commands.
 	* scsi.c: Add Mediavision CDR-H93MV to blacklist.
 Sat Oct 29 20:57:36 1994  Eric Youngdale  (eric@andante.aib.com)
 	* Linux 1.1.60 released.
 	* u14-34f.[c,h]: New driver from Dario Ballabio.
 	* aic7770.c, aha274x_seq.h, aha274x.seq, aha274x.h, aha274x.c,
 	README.aha274x: New files, new driver from John Aycock.
 Tue Oct 11 08:47:39 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.54 released.
 	* Add third PCI chip id.  [Drew]
 	* buslogic.c: Set BUSLOGIC_CMDLUN back to 1 [Eric].
 	* ultrastor.c: Fix asm directives for new GCC.
 	* sr.c, sd.c: Use new end_scsi_request function.
 	* scsi.h(end_scsi_request): Return pointer to block if still
 	active, else return NULL if inactive.  Fixes race condition.
 Sun Oct  9 20:23:14 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.53 released.
 	* scsi.c: Do not allocate dma bounce buffers if we have exactly
 	16Mb.
 Fri Sep  9 05:35:30 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.51 released.
 	* aha152x.c: Add support for disabling the parity check.  Update
 	to version 1.4. [Juergen].
 	* seagate.c: Tweak debugging message.
 Wed Aug 31 10:15:55 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.50 released.
 	* aha152x.c: Add eb800 for Vtech Platinum SMP boards. [Juergen].
 	* scsi.c: Add Quantum PD1225S to blacklist.
 Fri Aug 26 09:38:45 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.49 released.
 	* sd.c: Fix bug when we were deleting the wrong entry if we
 	get an unsupported sector size device.
 	* sr.c: Another spelling patch.
 Thu Aug 25 09:15:27 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.48 released.
 	* Throughout: Use new semantics for request_dma, as appropriate.
 	* sr.c: Print correct device number.
 Sun Aug 21 17:49:23 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.47 released.
 	* NCR5380.c: Add support for LIMIT_TRANSFERSIZE.
 	* constants.h: Add prototype for print_Scsi_Cmnd.
 	* pas16.c: Some more minor tweaks.  Test for Mediavision board.
 	Allow for disks > 1Gb.  [Drew??]
 	* sr.c: Set SCpnt->transfersize.
 Tue Aug 16 17:29:35 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.46 released.
 	* Throughout: More spelling fixups.
 	* buslogic.c: Add a few more fixups from Dave.  Disk translation
 	mainly.
 	* pas16.c: Add a few patches (Drew?).
 Thu Aug 11 20:45:15 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.44 released.
 	* hosts.c: Add type casts for scsi_init_malloc.
 	* scsicam.c: Add type cast.
 Wed Aug 10 19:23:01 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.43 released.
 	* Throughout: Spelling cleanups. [??]
 	* aha152x.c, NCR53*.c, fdomain.c, g_NCR5380.c, pas16.c, seagate.c,
 	 t128.c: Use request_irq, not irqaction. [??]
 	* aha1542.c: Move test for shost before we start to use shost.
 	* aha1542.c, aha1740.c, ultrastor.c, wd7000.c: Use new
 	calling sequence for request_irq.
 	* buslogic.c: Update from Dave Gentzel.
 Tue Aug  9 09:32:59 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.42 released.
 	* NCR5380.c: Change NCR5380_print_status to static.
 	* seagate.c: A few more bugfixes.  Only Drew knows what they are
 	for.
 	* ultrastor.c: Tweak some __asm__ directives so that it works
 	with newer compilers. [??]
 Sat Aug  6 21:29:36 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.40 released.
 	* NCR5380.c: Return SCSI_RESET_WAKEUP from reset function.
 	* aha1542.c: Reset mailbox status after a bus device reset.
 	* constants.c: Fix typo (;;).
 	* g_NCR5380.c:
 	* pas16.c:  Correct usage of NCR5380_init.
 	* scsi.c: Remove redundant (and unused variables).
 	* sd.c: Use memset to clear all of rscsi_disks before we use it.
 	* sg.c: Ditto, except for scsi_generics.
 	* sr.c: Ditto, except for scsi_CDs.
 	* st.c: Initialize STp->device.
 	* seagate.c: Fix bug. [Drew]
 Thu Aug  4 08:47:27 1994  Eric Youngdale  (eric@andante)
 	* Linux 1.1.39 released.
 	* Makefile: Fix typo in NCR53C7xx.
 	* st.c: Print correct number for device.
 Tue Aug  2 11:29:14 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.38 released.
 	Lots of changes in 1.1.38.  All from Drew unless otherwise noted.
 	* 53c7,8xx.c: New file from Drew.  PCI driver.
 	* 53c7,8xx.h: Likewise.
 	* 53c7,8xx.scr: Likewise.
 	* 53c8xx_d.h, 53c8xx_u.h, script_asm.pl: Likewise.
 	* scsicam.c: New file from Drew.  Read block 0 on the disk and
 	read the partition table.  Attempt to deduce the geometry from
 	the partition table if possible.  Only used by 53c[7,8]xx right
 	now, but could be used by any device for which we have no way
 	of identifying the geometry.
 	* sd.c: Use device letters instead of sd%d in a lot of messages.
 	* seagate.c: Fix bug that resulted in lockups with some devices.
 	* sr.c (sr_open): Return -EROFS, not -EACCES if we attempt to open
 	device for write.
 	* hosts.c, Makefile: Update for new driver.
 	* NCR5380.c, NCR5380.h, g_NCR5380.h: Update from Drew to support
 	53C400 chip.
 	* constants.c: Define CONST_CMND and CONST_MSG.  Other minor
 	cleanups along the way.  Improve handling of CONST_MSG.
 	* fdomain.c, fdomain.h: New version from Rik Faith.  Update to
 	5.18.  Should now support TMC-3260 PCI card with 18C30 chip.
 	* pas16.c: Update with new irq initialization.
 	* t128.c: Update with minor cleanups.
 	* scsi.c (scsi_pid): New variable - gives each command a unique
 	id. Add Quantum LPS5235S to blacklist.  Change in_scan to
 	in_scan_scsis and make global.
 	* scsi.h: Add some defines for extended message handling,
 	INITIATE/RELEASE_RECOVERY.  Add a few new fields to support sync
 	transfers.
 	* scsi_ioctl.h: Add ioctl to request synchronous transfers.
 Tue Jul 26 21:36:58 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.37 released.
 	* aha1542.c: Always call aha1542_mbenable, use new udelay
 	mechanism so we do not wait a long time if the board does not
 	implement this command.
 	* g_NCR5380.c: Remove #include <linux/config.h> and #if
 	defined(CONFIG_SCSI_*).
 	* seagate.c: Likewise.
 	Next round of changes to support loadable modules.  Getting closer
 	now, still not possible to do anything remotely usable.
 	hosts.c: Create a linked list of detected high level devices.
 	(scsi_register_device): New function to insert into this list.
 	(scsi_init): Call scsi_register_device for each of the known high
 	level drivers.
 	hosts.h: Add prototype for linked list header.  Add structure
 	definition for device template structure which defines the linked
 	list.
 	scsi.c: (scan_scsis): Use linked list instead of knowledge about
 	existing high level device drivers.
 	(scsi_dev_init): Use init functions for drivers on linked list
 	instead of explicit list to initialize and attach devices to high
 	level drivers.
 	scsi.h: Add new field "attached" to scsi_device - count of number
 	of high level devices attached.
 	sd.c, sr.c, sg.c, st.c: Adjust init/attach functions to use new
 	scheme.
 Sat Jul 23 13:03:17 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.35 released.
 	* ultrastor.c: Change constraint on asm() operand so that it works
 	with gcc 2.6.0.
 Thu Jul 21 10:37:39 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.33 released.
 	* sr.c(sr_open): Do not allow opens with write access.
-Mon Jul 18 09:51:22 1994 1994  Eric Youngdale  (eric@esp22)
+Mon Jul 18 09:51:22 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.31 released.
 	* sd.c: Increase SD_TIMEOUT from 300 to 600.
 	* sr.c: Remove stray task_struct* variable that was no longer
 	used.
 	* sr_ioctl.c: Fix typo in up() call.
 Sun Jul 17 16:25:29 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.30 released.
 	* scsi.c (scan_scsis): Fix detection of some Toshiba CDROM drives
 	that report themselves as disk drives.
 	* (Throughout): Use request.sem instead of request.waiting.
 	Should fix swap problem with fdomain.
 Thu Jul 14 10:51:42 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.29 released.
 	* scsi.c (scan_scsis): Add new devices to end of linked list, not
 	to the beginning.
 	* scsi.h (SCSI_SLEEP): Remove brain dead hack to try to save
 	the task state before sleeping.
 Sat Jul  9 15:01:03 1994  Eric Youngdale  (eric@esp22)
 	More changes to eventually support loadable modules.  Mainly
 	we want to use linked lists instead of arrays because it is easier
 	to dynamically add and remove things this way.
 	Quite a bit more work is needed before loadable modules are
 	possible (and usable) with scsi, but this is most of the grunge
 	work.
 	* Linux 1.1.28 released.
 	* scsi.c, scsi.h (allocate_device, request_queueable): Change
 	argument from index into scsi_devices to a pointer to the
 	Scsi_Device struct.
 	* Throughout: Change all calls to allocate_device,
 	request_queueable to use new calling sequence.
 	* Throughout: Use SCpnt->device instead of
 	scsi_devices[SCpnt->index].  Ugh - the pointer was there all along
 	- much cleaner this way.
 	* scsi.c (scsi_init_malloc, scsi_free_malloc): New functions -
 	allow us to pretend that we have a working malloc when we
 	initialize.  Use this instead of passing memory_start, memory_end
 	around all over the place.
 	* scsi.h, st.c, sr.c, sd.c, sg.c: Change *_init1 functions to use
 	scsi_init_malloc, remove all arguments, no return value.
 	* scsi.h: Remove index field from Scsi_Device and Scsi_Cmnd
 	structs.
 	* scsi.c (scsi_dev_init): Set up for scsi_init_malloc.
 	(scan_scsis): Get SDpnt from scsi_init_malloc, and refresh
 	when we discover a device.  Free pointer before returning.
 	Change scsi_devices into a linked list.
 	* scsi.c (scan_scsis): Change to only scan one host.
 	(scsi_dev_init): Loop over all detected hosts, and scan them.
 	* hosts.c  (scsi_init_free): Change so that  number of extra bytes
 	is stored in struct, and we do not have to pass it each time.
 	* hosts.h: Change Scsi_Host_Template struct to include "next" and
 	"release" functions.  Initialize to NULL in all low level
 	adapters.
 	* hosts.c: Rename scsi_hosts to builtin_scsi_hosts, create linked
 	list scsi_hosts, linked together with the new "next" field.
 Wed Jul  6 05:45:02 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.25 released.
 	* aha152x.c: Changes from Juergen - cleanups and updates.
 	* sd.c, sr.c: Use new check_media_change and revalidate
 	file_operations fields.
 	* st.c, st.h: Add changes from Kai Makisara, dated Jun 22.
 	* hosts.h: Change SG_ALL back to 0xff.  Apparently soft error
 	in /dev/brain resulted in having this bumped up.
 	Change first parameter in bios_param function to be Disk * instead
 	of index into rscsi_disks.
 	* sd_ioctl.c: Pass pointer to rscsi_disks element instead of index
 	to array.
 	* sd.h: Add struct name "scsi_disk" to typedef for Scsi_Disk.
 	* scsi.c: Remove redundant Maxtor XT8760S from blacklist.
 	In scsi_reset, add printk when DEBUG defined.
 	* All low level drivers: Modify definitions of bios_param in
 	appropriate way.
 Thu Jun 16 10:31:59 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.20 released.
 	* scsi_ioctl.c: Only pass down the actual number of characters
 	required to scsi_do_cmd, not the one rounded up to a even number
 	of sectors.
 	* ultrastor.c: Changes from Caleb Epstein for 24f cards.  Support
 	larger SG lists.
 	* ultrastor.c: Changes from me - use scsi_register to register
 	host.  Add some consistency checking,
 Wed Jun  1 21:12:13 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.19 released.
 	* scsi.h: Add new return code for reset() function:
 	SCSI_RESET_PUNT.
 	* scsi.c: Make SCSI_RESET_PUNT the same as SCSI_RESET_WAKEUP for
 	now.
 	* aha1542.c: If the command responsible for the reset is not
 	pending, return SCSI_RESET_PUNT.
 	* aha1740.c, buslogic.c, wd7000.c, ultrastor.c: Return
 	SCSI_RESET_PUNT instead of SCSI_RESET_SNOOZE.
 Tue May 31 19:36:01 1994  Eric Youngdale  (eric@esp22)
 	* buslogic.c: Do not print out message about "must be Adaptec"
 	if we have detected a buslogic card.  Print out a warning message
 	if we are configuring for >16Mb, since the 445S at board level
 	D or earlier does not work right.  The "D" level board can be made
 	to work by flipping an undocumented switch, but this is too subtle.
    Changes based upon patches in Yggdrasil distribution.
 	* sg.c, sg.h: Return sense data to user.
 	* aha1542.c, aha1740.c, buslogic.c: Do not panic if
 	sense buffer is wrong size.
 	* hosts.c: Test for ultrastor card before any of the others.
 	* scsi.c: Allow boot-time option for max_scsi_luns=? so that
 	buggy firmware has an easy work-around.
 Sun May 15 20:24:34 1994  Eric Youngdale  (eric@esp22)
 	* Linux 1.1.15 released.
 	Post-codefreeze thaw...
 	* buslogic.[c,h]: New driver from David Gentzel.
 	* hosts.h: Add use_clustering field to explicitly say whether
 	clustering should be used for devices attached to this host
 	adapter.  The buslogic board apparently supports large SG lists,
 	but it is apparently faster if sd.c condenses this into a smaller
 	list.
 	* sd.c: Use this field instead of heuristic.
 	* All host adapter include files: Add appropriate initializer for
 	use_clustering field.
 	* scsi.h: Add #defines for return codes for the abort and reset
 	functions.  There are now a specific set of return codes to fully
 	specify all of the possible things that the low-level adapter
 	could do.
 	* scsi.c: Act based upon return codes from abort/reset functions.
 	* All host adapter abort/reset functions: Return new return code.
 	* Add code in scsi.c to help debug timeouts.  Use #define
 	DEBUG_TIMEOUT to enable this.
 	* scsi.c: If the host->irq field is set, use
 	disable_irq/enable_irq before calling queuecommand if we
 	are not already in an interrupt.  Reduce races, and we
 	can be sloppier about cli/sti in the interrupt routines now
 	(reduce interrupt latency).
 	* constants.c: Fix some things to eliminate warnings.  Add some
 	sense descriptions that were omitted before.
 	* aha1542.c: Watch for SCRD from host adapter - if we see it, set
 	a flag.  Currently we only print out the number of pending
 	commands that might need to be restarted.
 	* aha1542.c (aha1542_abort): Look for lost interrupts, OGMB still
 	full, and attempt to recover.  Otherwise give up.
 	* aha1542.c (aha1542_reset): Try BUS DEVICE RESET, and then pass
 	DID_RESET back up to the upper level code for all commands running
 	on this target (even on different LUNs).
 Sat May  7 14:54:01 1994
 	* Linux 1.1.12 released.
 	* st.c, st.h: New version from Kai.  Supports boot time
 	specification of number of buffers.
 	* wd7000.[c,h]: Updated driver from John Boyd.  Now supports
 	more than one wd7000 board in machine at one time, among other things.
 Wed Apr 20 22:20:35 1994
 	* Linux 1.1.8 released.
 	* sd.c: Add a few type casts where scsi_malloc is called.
 Wed Apr 13 12:53:29 1994
 	* Linux 1.1.4 released.
 	* scsi.c: Clean up a few printks (use %p to print pointers).
 Wed Apr 13 11:33:02 1994
 	* Linux 1.1.3 released.
 	* fdomain.c: Update to version 5.16 (Handle different FIFO sizes
 	better).
 Fri Apr  8 08:57:19 1994
 	* Linux 1.1.2 released.
 	* Throughout: SCSI portion of cluster diffs added.
 Tue Apr  5 07:41:50 1994
 	* Linux 1.1 development tree initiated.
 	* The linux 1.0 development tree is now effectively frozen except
 	for obvious bugfixes.
 ******************************************************************
 ******************************************************************
 ******************************************************************
 ******************************************************************
 Sun Apr 17 00:17:39 1994
 	* Linux 1.0, patchlevel 9 released.
 	* fdomain.c: Update to version 5.16 (Handle different FIFO sizes
 	better).
 Thu Apr  7 08:36:20 1994
 	* Linux 1.0, patchlevel8 released.
 	* fdomain.c: Update to version 5.15 from 5.9.  Handles 3.4 bios.
 Sun Apr  3 14:43:03 1994
 	* Linux 1.0, patchlevel6 released.
 	* wd7000.c: Make stab at fixing race condition.
 Sat Mar 26 14:14:50 1994
 	* Linux 1.0, patchlevel5 released.
 	* aha152x.c, Makefile: Fix a few bugs (too much data message).
 	Add a few more bios signatures. (Patches from Juergen).
 	* aha1542.c: Fix race condition in aha1542_out.
 Mon Mar 21 16:36:20 1994
 	* Linux 1.0, patchlevel3 released.
 	* sd.c, st.c, sr.c, sg.c: Return -ENXIO, not -ENODEV if we attempt
 	to open a non-existent device.
 	* scsi.c: Add Chinon cdrom to blacklist.
 	* sr_ioctl.c: Check return status of verify_area.
 Sat Mar  6 16:06:19 1994
 	* Linux 1.0 released (technically a pre-release).
 	* scsi.c: Add IMS CDD521, Maxtor XT-8760S to blacklist.
 Tue Feb 15 10:58:20 1994
        * pl15e released.
        * aha1542.c: For 1542C, allow dynamic device scan with >1Gb turned
 	off.
 	* constants.c: Fix typo in definition of CONSTANTS.
        * pl15d released.
 Fri Feb 11 10:10:16 1994
        * pl15c released.
 	* scsi.c: Add Maxtor XT-3280 and Rodime RO3000S to blacklist.
 	* scsi.c: Allow tagged queueing for scsi 3 devices as well.
 	Some really old devices report a version number of 0.  Disallow
 	LUN != 0 for these.
 Thu Feb 10 09:48:57 1994
        * pl15b released.
 Sun Feb  6 12:19:46 1994
        * pl15a released.
 Fri Feb  4 09:02:17 1994
        * scsi.c: Add Teac cdrom to blacklist.
 Thu Feb  3 14:16:43 1994
 	* pl15 released.
 Tue Feb  1 15:47:43 1994
 	* pl14w released.
 	* wd7000.c (wd_bases): Fix typo in last change.
 Mon Jan 24 17:37:23 1994
 	* pl14u released.
 	* aha1542.c: Support 1542CF/extended bios.  Different from 1542C
 	* wd7000.c: Allow bios at 0xd8000 as well.
 	* ultrastor.c: Do not truncate cylinders to 1024.
 	* fdomain.c: Update to version 5.9 (add new bios signature).
 	* NCR5380.c: Update from Drew - should work a lot better now.
 Sat Jan  8 15:13:10 1994
 	* pl14o released.
 	* sr_ioctl.c: Zero reserved field before trying to set audio volume.
 Wed Jan  5 13:21:10 1994
 	* pl14m released.
 	* fdomain.c: Update to version 5.8.  No functional difference???
 Tue Jan  4 14:26:13 1994
 	* pl14l released.
 	* ultrastor.c: Remove outl, inl functions (now provided elsewhere).
 Mon Jan  3 12:27:25 1994
 	* pl14k released.
 	* aha152x.c: Remove insw and outsw functions.
 	* fdomain.c: Ditto.
 Wed Dec 29 09:47:20 1993
 	* pl14i released.
 	* scsi.c: Support RECOVERED_ERROR for tape drives.
 	* st.c: Update of tape driver from Kai.
 Tue Dec 21 09:18:30 1993
 	* pl14g released.
 	* aha1542.[c,h]: Support extended BIOS stuff.
 	* scsi.c: Clean up messages about disks, so they are displayed as
 	sda, sdb, etc instead of sd0, sd1, etc.
 	* sr.c:  Force reread of capacity if disk was changed.
 	Clear buffer before asking for capacity/sectorsize (some drives
 	do not report this properly).  Set needs_sector_size flag if
 	drive did not return sensible sector size.
 Mon Dec 13 12:13:47 1993
 	* aha152x.c: Update to version .101 from Juergen.
 Mon Nov 29 03:03:00 1993
        * linux 0.99.14 released.
 	* All scsi stuff moved from kernel/blk_drv/scsi to drivers/scsi.
 	* Throughout: Grammatical corrections to various comments.
 	* Makefile: fix so that we do not need to compile things we are
 	not going to use.
 	* NCR5380.c, NCR5380.h, g_NCR5380.c, g_NCR5380.h, pas16.c,
 	pas16.h, t128.c, t128.h:  New files from Drew.
 	* aha152x.c, aha152x.h: New files from Juergen Fischer.
 	* aha1542.c: Support for more than one 1542 in the machine
 	at the same time.  Make functions static that do not need
 	visibility.
 	* aha1740.c: Set NEEDS_JUMPSTART flag in reset function, so we
 	know to restart the command.  Change prototype of aha1740_reset
 	to take a command pointer.
 	* constants.c: Clean up a few things.
 	* fdomain.c: Update to version 5.6.  Move snarf_region.  Allow
 	board to be set at different SCSI ids.  Remove support for
 	reselection (did not work well).  Set JUMPSTART flag in reset
 	code.
 	* hosts.c: Support new low-level adapters.  Allow for more than
 	one adapter of a given type.
 	* hosts.h: Allow for more than one adapter of a given type.
 	* scsi.c:  Add scsi_device_types array, if NEEDS_JUMPSTART is set
 	after a low-level reset, start the command again.  Sort blacklist,
 	and add Maxtor MXT-1240S, XT-4170S, NEC CDROM 84, Seagate ST157N.
 	* scsi.h: Add constants for tagged queueing.
 	* Throughout: Use constants from major.h instead of hardcoded
 	numbers for major numbers.
 	* scsi_ioctl.c: Fix bug in buffer length in ioctl_command.  Use
 	verify_area in GET_IDLUN ioctl.  Add new ioctls for
 	TAGGED_QUEUE_ENABLE, DISABLE.  Only allow IOCTL_SEND_COMMAND by
 	superuser.
 	* sd.c: Only pay attention to UNIT_ATTENTION for removable disks.
 	Fix bug where sometimes portions of blocks would get lost
 	resulting in processes hanging.  Add messages when we spin up a
 	disk, and fix a bug in the timing.  Increase read-ahead for disks
 	that are on a scatter-gather capable host adapter.
 	* seagate.c: Fix so that some parameters can be set from the lilo
 	prompt.  Supply jumpstart flag if we are resetting and need the
 	command restarted.   Fix so that we return 1 if we detect a card
 	so that multiple card detection works correctly.  Add yet another
 	signature for FD cards (950).  Add another signature for ST0x.
 	* sg.c, sg.h: New files from Lawrence Foard for generic scsi
 	access.
 	* sr.c:  Add type casts for (void*) so that we can do pointer
 	arithmetic.  Works with GCC without this, but it is not strictly
 	correct.  Same bugfix as was in sd.c.  Increase read-ahead a la
 	disk driver.
 	* sr_ioctl.c: Use scsi_malloc buffer instead of buffer from stack
 	since we cannot guarantee that the stack is < 16Mb.
 	ultrastor.c: Update to support 24f properly (JFC's driver).
 	wd7000.c: Supply jumpstart flag for reset.  Do not round up
 	number of cylinders in biosparam function.
 Sat Sep  4 20:49:56 1993
    * 0.99pl13 released.
    * Throughout:  Use check_region/snarf_region for all low-level
    drivers.
    * aha1542.c: Do hard reset instead of soft (some ethercard probes
    screw us up).
    * scsi.c: Add new flag ASKED_FOR_SENSE so that we can tell if we are
    in a loop whereby the device returns null sense data.
    * sd.c: Add code to spin up a drive if it is not already spinning.
    Do this one at a time to make it easier on power supplies.
    * sd_ioctl.c: Use sync_dev instead of fsync_dev in BLKFLSBUF ioctl.
    * seagate.c: Switch around DATA/CONTROL lines.
    * st.c: Change sense to unsigned.
 Thu Aug  5 11:59:18 1993
    * 0.99pl12 released.
    * constants.c, constants.h: New files with ascii descriptions of
    various conditions.
    * Makefile: Do not try to count the number of low-level drivers,
    just generate the list of .o files.
    * aha1542.c: Replace 16 with sizeof(SCpnt->sense_buffer).  Add tests
    for addresses > 16Mb, panic if we find one.
    * aha1740.c: Ditto with sizeof().
    * fdomain.c: Update to version 3.18.  Add new signature, register IRQ
    with irqaction.  Use ID 7 for new board.  Be more intelligent about
    obtaining the h/s/c numbers for biosparam.
    * hosts.c: Do not depend upon Makefile generated count of the number
    of low-level host adapters.
    * scsi.c: Use array for scsi_command_size instead of a function.  Add
    Texel cdrom and Maxtor XT-4380S to blacklist.  Allow compile time
    option for no-multi lun scan.  Add semaphore for possible problems
    with handshaking, assume device is faulty until we know it not to be
    the case.  Add DEBUG_INIT symbol to dump info as we scan for devices.
    Zero sense buffer so we can tell if we need to request it.  When
    examining sense information, request sense if buffer is all zero.
    If RESET, request sense information to see what to do next.
    * scsi_debug.c: Change some constants to use symbols like INT_MAX.
    * scsi_ioctl.c (kernel_scsi_ioctl): New function -for making ioctl
    calls from kernel space.
    * sd.c: Increase timeout to 300.  Use functions in constants.h to
    display info.  Use scsi_malloc buffer for READ_CAPACITY, since
    we cannot guarantee that a stack based buffer is < 16Mb.
    * sd_ioctl.c: Add BLKFLSBUF ioctl.
    * seagate.c: Add new compile time options for ARBITRATE,
    SLOW_HANDSHAKE, and SLOW_RATE.  Update assembly loops for transferring
    data.  Use kernel_scsi_ioctl to request mode page with geometry.
    * sr.c: Use functions in constants.c to display messages.
    * st.c: Support for variable block size.
    * ultrastor.c: Do not use cache for tape drives.  Set
    unchecked_isa_dma flag, even though this may not be needed (gets set
    later).
 Sat Jul 17 18:32:44 1993
    * 0.99pl11 released.  C++ compilable.
    * Throughout: Add type casts all over the place, and use "ip" instead
    of "info" in the various biosparam functions.
    * Makefile: Compile seagate.c with C++ compiler.
    * aha1542.c: Always set ccb pointer as this gets trashed somehow on
    some systems.  Add a few type casts.  Update biosparam function a little.
    * aha1740.c: Add a few type casts.
    * fdomain.c: Update to version 3.17 from 3.6.  Now works with
    TMC-18C50.
    * scsi.c: Minor changes here and there with datatypes.  Save use_sg
    when requesting sense information so that this can properly be
    restored if we retry the command.  Set aside dma buffers assuming each
    block is 1 page, not 1Kb minix block.
    * scsi_ioctl.c: Add a few type casts.  Other minor changes.
    * sd.c:  Correctly  free all scsi_malloc'd memory if we run out of
    dma_pool. Store blocksize information for each partition.
    * seagate.c: Minor cleanups here and there.
    * sr.c: Set up blocksize array for all discs.  Fix bug in freeing
    buffers if we run out of dma pool.
 Thu Jun  2 17:58:11 1993
    * 0.99pl10 released.
    * aha1542.c: Support for BT 445S (VL-bus board with no dma channel).
    * fdomain.c: Upgrade to version 3.6. Preliminary support for TNC-18C50.
    * scsi.c: First attempt to fix problem with old_use_sg.  Change
    NOT_READY to a SUGGEST_ABORT.  Fix timeout race where time might
    get decremented past zero.
    * sd.c: Add block_fsync function to dispatch table.
    * sr.c: Increase timeout to 500 from 250.  Add entry for sync in
    dispatch table (supply NULL).  If we do not have a sectorsize,
    try to get it in the sd_open function.  Add new function just to
    obtain sectorsize.
    * sr.h: Add needs_sector_size semaphore.
    * st.c: Add NULL for fsync in dispatch table.
    * wd7000.c: Allow another condition for power on that are normal
    and do not require a panic.
 Thu Apr 22 23:10:11 1993
    * 0.99pl9 released.
    * aha1542.c: Use (void) instead of () in setup_mailboxes.
    * scsi.c: Initialize transfersize and underflow fields in SCmd to 0.
    Do not panic for unsupported message bytes.
    * scsi.h: Allocate 12 bytes instead of 10 for commands.  Add
    transfersize and underflow fields.
    * scsi_ioctl.c: Further bugfix to ioctl_probe.
    * sd.c: Use long instead of int for last parameter in sd_ioctl.
    Initialize transfersize and underflow fields.
    * sd_ioctl.c: Ditto for sd_ioctl(,,,,);
    * seagate.c: New version from Drew.  Includes new signatures for FD
    cards.  Support for 0ws jumper. Correctly initialize
    scsi_hosts[hostnum].this_id.  Improved handing of
    disconnect/reconnect, and support command linking.  Use
    transfersize and underflow fields.  Support scatter-gather.
    * sr.c, sr_ioctl.c: Use long instead of int for last parameter in sr_ioctl.
    Use buffer and buflength in do_ioctl.  Patches from Chris Newbold for
    scsi-2 audio commands.
    * ultrastor.c: Comment out in_byte (compiler warning).
    * wd7000.c: Change () to (void) in wd7000_enable_dma.
 Wed Mar 31 16:36:25 1993
    * 0.99pl8 released.
    * aha1542.c: Handle mailboxes better for 1542C.
        Do not truncate number of cylinders at 1024 for biosparam call.
    * aha1740.c: Fix a few minor bugs for multiple devices.
        Same as above for biosparam.
    * scsi.c: Add lockable semaphore for removable devices that can have
    media removal prevented.  Add another signature for flopticals.
    (allocate_device): Fix race condition.  Allow more space in dma pool
    for blocksizes of up to 4Kb.
    * scsi.h: Define COMMAND_SIZE.  Define a SCSI specific version of
    INIT_REQUEST that can run with interrupts off.
    * scsi_ioctl.c: Make ioctl_probe function more idiot-proof.  If
    a removable device says ILLEGAL REQUEST to a door-locking command,
    clear lockable flag.  Add SCSI_IOCTL_GET_IDLUN ioctl.  Do not attempt
    to lock door for devices that do not have lockable semaphore set.
    * sd.c: Fix race condition for multiple disks.  Use INIT_SCSI_REQUEST
    instead of INIT_REQUEST.  Allow sector sizes of 1024 and 256.  For
    removable disks that are not ready, mark them as having a media change
    (some drives do not report this later).
    * seagate.c: Use volatile keyword for memory-mapped register pointers.
    * sr.c: Fix race condition, a la sd.c.  Increase the number of retries
    to 1.  Use INIT_SCSI_REQUEST.  Allow 512 byte sector sizes.  Do a
    read_capacity when we init the device so we know the size and
    sectorsize.
    * st.c: If ioctl not found in st.c, try scsi_ioctl for others.
    * ultrastor.c: Do not truncate number of cylinders at 1024 for
    biosparam call.
    * wd7000.c: Ditto.
    Throughout: Use COMMAND_SIZE macro to determine length of scsi
    command.
 Sat Mar 13 17:31:29 1993
    * 0.99pl7 released.
    Throughout: Improve punctuation in some messages, and use new
    verify_area syntax.
    * aha1542.c: Handle unexpected interrupts better.
    * scsi.c: Ditto.  Handle reset conditions a bit better, asking for
    sense information and retrying if required.
    * scsi_ioctl.c: Allow for 12 byte scsi commands.
    * ultrastor.c: Update to use scatter-gather.
 Sat Feb 20 17:57:15 1993
    * 0.99pl6 released.
    * fdomain.c: Update to version 3.5.  Handle spurious interrupts
    better.
    * sd.c: Use register_blkdev function.
    * sr.c: Ditto.
    * st.c: Use register_chrdev function.
    * wd7000.c: Undo previous change.
 Sat Feb  6 11:20:43 1993
    * 0.99pl5 released.
    * scsi.c: Fix bug in testing for UNIT_ATTENTION.
    * wd7000.c: Check at more addresses for bios.  Fix bug in biosparam
    (heads & sectors turned around).
 Wed Jan 20 18:13:59 1993
    * 0.99pl4 released.
    * scsi.c: Ignore leading spaces when looking for blacklisted devices.
    * seagate.c: Add a few new signatures for FD cards.  Another patch
    with SCint to fix race condition.  Use recursion_depth to keep track
    of how many times we have been recursively called, and do not start
    another command unless we are on the outer level.  Fixes bug
    with Syquest cartridge drives (used to crash kernel), because
    they do not disconnect with large data transfers.
 Tue Jan 12 14:33:36 1993
    * 0.99pl3 released.
    * fdomain.c: Update to version 3.3 (a few new signatures).
    * scsi.c: Add CDU-541, Denon DRD-25X to blacklist.
    (allocate_request, request_queueable): Init request.waiting to NULL if
    non-buffer type of request.
    * seagate.c:  Allow controller to be overridden with CONTROLLER symbol.
    Set SCint=NULL when we are done, to remove race condition.
    * st.c: Changes from Kai.
 Wed Dec 30 20:03:47 1992
    * 0.99pl2 released.
    * scsi.c: Blacklist back in.  Remove Newbury drive as other bugfix
    eliminates need for it here.
    * sd.c: Return ENODEV instead of EACCES if no such device available.
    (sd_init) Init blkdev_fops earlier so that sd_open is available sooner.
    * sr.c: Same as above for sd.c.
    * st.c: Return ENODEV instead of ENXIO if no device.  Init chrdev_fops
    sooner, so that it is always there even if no tapes.
    * seagate.c (controller_type): New variable to keep track of ST0x or
    FD.  Modify signatures list to indicate controller type, and init
    controller_type once we find a match.
    * wd7000.c (wd7000_set_sync): Remove redundant function.
 Sun Dec 20 16:26:24 1992
    * 0.99pl1 released.
    * scsi_ioctl.c: Bugfix - check dev->index, not dev->id against
    NR_SCSI_DEVICES.
    * sr_ioctl.c: Verify that device exists before allowing an ioctl.
    * st.c: Patches from Kai - change timeout values, improve end of tape
    handling.
 Sun Dec 13 18:15:23 1992
    * 0.99 kernel released.  Baseline for this ChangeLog.
 This file contains brief information about the SCSI tape driver.
 The driver is currently maintained by Kai Mäkisara (email
 Kai.Makisara@kolumbus.fi)
 Last modified: Mon Mar  7 21:14:44 2005 by kai.makisara
 BASICS
 The driver is generic, i.e., it does not contain any code tailored
 to any specific tape drive. The tape parameters can be specified with
 one of the following three methods:
 1. Each user can specify the tape parameters he/she wants to use
 directly with ioctls. This is administratively a very simple and
 flexible method and applicable to single-user workstations. However,
 in a multiuser environment the next user finds the tape parameters in
 state the previous user left them.
 2. The system manager (root) can define default values for some tape
 parameters, like block size and density using the MTSETDRVBUFFER ioctl.
 These parameters can be programmed to come into effect either when a
 new tape is loaded into the drive or if writing begins at the
 beginning of the tape. The second method is applicable if the tape
 drive performs auto-detection of the tape format well (like some
 QIC-drives). The result is that any tape can be read, writing can be
 continued using existing format, and the default format is used if
 the tape is rewritten from the beginning (or a new tape is written
 for the first time). The first method is applicable if the drive
 does not perform auto-detection well enough and there is a single
 "sensible" mode for the device. An example is a DAT drive that is
 used only in variable block mode (I don't know if this is sensible
 or not :-).
 The user can override the parameters defined by the system
 manager. The changes persist until the defaults again come into
 effect.
 3. By default, up to four modes can be defined and selected using the minor
 number (bits 5 and 6). The number of modes can be changed by changing
 ST_NBR_MODE_BITS in st.h. Mode 0 corresponds to the defaults discussed
 above. Additional modes are dormant until they are defined by the
 system manager (root). When specification of a new mode is started,
 the configuration of mode 0 is used to provide a starting point for
 definition of the new mode.
 Using the modes allows the system manager to give the users choices
 over some of the buffering parameters not directly accessible to the
 users (buffered and asynchronous writes). The modes also allow choices
 between formats in multi-tape operations (the explicitly overridden
 parameters are reset when a new tape is loaded).
 If more than one mode is used, all modes should contain definitions
 for the same set of parameters.
 Many Unices contain internal tables that associate different modes to
 supported devices. The Linux SCSI tape driver does not contain such
 tables (and will not do that in future). Instead of that, a utility
 program can be made that fetches the inquiry data sent by the device,
 scans its database, and sets up the modes using the ioctls. Another
 alternative is to make a small script that uses mt to set the defaults
 tailored to the system.
 The driver supports fixed and variable block size (within buffer
 limits). Both the auto-rewind (minor equals device number) and
 non-rewind devices (minor is 128 + device number) are implemented.
 In variable block mode, the byte count in write() determines the size
 of the physical block on tape. When reading, the drive reads the next
 tape block and returns to the user the data if the read() byte count
 is at least the block size. Otherwise, error ENOMEM is returned.
 In fixed block mode, the data transfer between the drive and the
 driver is in multiples of the block size. The write() byte count must
 be a multiple of the block size. This is not required when reading but
 may be advisable for portability.
 Support is provided for changing the tape partition and partitioning
 of the tape with one or two partitions. By default support for
 partitioned tape is disabled for each driver and it can be enabled
 with the ioctl MTSETDRVBUFFER.
 By default the driver writes one filemark when the device is closed after
 writing and the last operation has been a write. Two filemarks can be
 optionally written. In both cases end of data is signified by
 returning zero bytes for two consecutive reads.
 If rewind, offline, bsf, or seek is done and previous tape operation was
 write, a filemark is written before moving tape.
 The compile options are defined in the file linux/drivers/scsi/st_options.h.
 4. If the open option O_NONBLOCK is used, open succeeds even if the
 drive is not ready. If O_NONBLOCK is not used, the driver waits for
 the drive to become ready. If this does not happen in ST_BLOCK_SECONDS
 seconds, open fails with the errno value EIO. With O_NONBLOCK the
 device can be opened for writing even if there is a write protected
 tape in the drive (commands trying to write something return error if
 attempted).
 MINOR NUMBERS
 The tape driver currently supports 128 drives by default. This number
 can be increased by editing st.h and recompiling the driver if
 necessary. The upper limit is 2^17 drives if 4 modes for each drive
 are used.
 The minor numbers consist of the following bit fields:
 dev_upper non-rew mode dev-lower
  20 -  8     7    6 5  4      0
 The non-rewind bit is always bit 7 (the uppermost bit in the lowermost
 byte). The bits defining the mode are below the non-rewind bit. The
 remaining bits define the tape device number. This numbering is
 backward compatible with the numbering used when the minor number was
 only 8 bits wide.
 SYSFS SUPPORT
 The driver creates the directory /sys/class/scsi_tape and populates it with
 directories corresponding to the existing tape devices. There are autorewind
 and non-rewind entries for each mode. The names are stxy and nstxy, where x
 is the tape number and y a character corresponding to the mode (none, l, m,
 a). For example, the directories for the first tape device are (assuming four
 modes): st0  nst0  st0l  nst0l  st0m  nst0m  st0a  nst0a.
 Each directory contains the entries: default_blksize  default_compression
 default_density  defined  dev  device  driver. The file 'defined' contains 1
 if the mode is defined and zero if not defined. The files 'default_*' contain
 the defaults set by the user. The value -1 means the default is not set. The
 file 'dev' contains the device numbers corresponding to this device. The links
 'device' and 'driver' point to the SCSI device and driver entries.
 A link named 'tape' is made from the SCSI device directory to the class
 directory corresponding to the mode 0 auto-rewind device (e.g., st0). 
 BSD AND SYS V SEMANTICS
 The user can choose between these two behaviours of the tape driver by
 defining the value of the symbol ST_SYSV. The semantics differ when a
 file being read is closed. The BSD semantics leaves the tape where it
 currently is whereas the SYS V semantics moves the tape past the next
 filemark unless the filemark has just been crossed.
 The default is BSD semantics.
 BUFFERING
 The driver tries to do transfers directly to/from user space. If this
 is not possible, a driver buffer allocated at run-time is used. If
 direct i/o is not possible for the whole transfer, the driver buffer
 is used (i.e., bounce buffers for individual pages are not
 used). Direct i/o can be impossible because of several reasons, e.g.:
 - one or more pages are at addresses not reachable by the HBA
 - the number of pages in the transfer exceeds the number of
  scatter/gather segments permitted by the HBA
 - one or more pages can't be locked into memory (should not happen in
  any reasonable situation)
 The size of the driver buffers is always at least one tape block. In fixed
 block mode, the minimum buffer size is defined (in 1024 byte units) by
 ST_FIXED_BUFFER_BLOCKS. With small block size this allows buffering of
 several blocks and using one SCSI read or write to transfer all of the
 blocks. Buffering of data across write calls in fixed block mode is
 allowed if ST_BUFFER_WRITES is non-zero and direct i/o is not used.
 Buffer allocation uses chunks of memory having sizes 2^n * (page
 size). Because of this the actual buffer size may be larger than the
 minimum allowable buffer size.
 NOTE that if direct i/o is used, the small writes are not buffered. This may
 cause a surprise when moving from 2.4. There small writes (e.g., tar without
 -b option) may have had good throughput but this is not true any more with
 2.6. Direct i/o can be turned off to solve this problem but a better solution
 is to use bigger write() byte counts (e.g., tar -b 64).
 Asynchronous writing. Writing the buffer contents to the tape is
 started and the write call returns immediately. The status is checked
 at the next tape operation. Asynchronous writes are not done with
 direct i/o and not in fixed block mode.
 Buffered writes and asynchronous writes may in some rare cases cause
 problems in multivolume operations if there is not enough space on the
 tape after the early-warning mark to flush the driver buffer.
 Read ahead for fixed block mode (ST_READ_AHEAD). Filling the buffer is
 attempted even if the user does not want to get all of the data at
 this read command. Should be disabled for those drives that don't like
 a filemark to truncate a read request or that don't like backspacing.
 Scatter/gather buffers (buffers that consist of chunks non-contiguous
 in the physical memory) are used if contiguous buffers can't be
 allocated. To support all SCSI adapters (including those not
 supporting scatter/gather), buffer allocation is using the following
 three kinds of chunks:
 1. The initial segment that is used for all SCSI adapters including
 those not supporting scatter/gather. The size of this buffer will be
 (PAGE_SIZE << ST_FIRST_ORDER) bytes if the system can give a chunk of
 this size (and it is not larger than the buffer size specified by
 ST_BUFFER_BLOCKS). If this size is not available, the driver halves
 the size and tries again until the size of one page. The default
 settings in st_options.h make the driver to try to allocate all of the
 buffer as one chunk.
 2. The scatter/gather segments to fill the specified buffer size are
 allocated so that as many segments as possible are used but the number
 of segments does not exceed ST_FIRST_SG.
 3. The remaining segments between ST_MAX_SG (or the module parameter
 max_sg_segs) and the number of segments used in phases 1 and 2
 are used to extend the buffer at run-time if this is necessary. The
 number of scatter/gather segments allowed for the SCSI adapter is not
 exceeded if it is smaller than the maximum number of scatter/gather
 segments specified. If the maximum number allowed for the SCSI adapter
 is smaller than the number of segments used in phases 1 and 2,
 extending the buffer will always fail.
 EOM BEHAVIOUR WHEN WRITING
 When the end of medium early warning is encountered, the current write
 is finished and the number of bytes is returned. The next write
 returns -1 and errno is set to ENOSPC. To enable writing a trailer,
 the next write is allowed to proceed and, if successful, the number of
 bytes is returned. After this, -1 and the number of bytes are
 alternately returned until the physical end of medium (or some other
 error) is encountered.
 MODULE PARAMETERS
 The buffer size, write threshold, and the maximum number of allocated buffers
 are configurable when the driver is loaded as a module. The keywords are:
 buffer_kbs=xxx             the buffer size for fixed block mode is set
 			   to xxx kilobytes
 write_threshold_kbs=xxx    the write threshold in kilobytes set to xxx
 max_sg_segs=xxx		   the maximum number of scatter/gather
 			   segments
 try_direct_io=x		   try direct transfer between user buffer and
 			   tape drive if this is non-zero
 Note that if the buffer size is changed but the write threshold is not
 set, the write threshold is set to the new buffer size - 2 kB.
 BOOT TIME CONFIGURATION
 If the driver is compiled into the kernel, the same parameters can be
 also set using, e.g., the LILO command line. The preferred syntax is
-is to use the same keyword used when loading as module but prepended
+to use the same keyword used when loading as module but prepended
 with 'st.'. For instance, to set the maximum number of scatter/gather
 segments, the parameter 'st.max_sg_segs=xx' should be used (xx is the
 number of scatter/gather segments).
 For compatibility, the old syntax from early 2.5 and 2.4 kernel
 versions is supported. The same keywords can be used as when loading
 the driver as module. If several parameters are set, the keyword-value
 pairs are separated with a comma (no spaces allowed). A colon can be
 used instead of the equal mark. The definition is prepended by the
 string st=. Here is an example:
 	st=buffer_kbs:64,write_threhold_kbs:60
 The following syntax used by the old kernel versions is also supported:
           st=aa[,bb[,dd]]
 where
  aa is the buffer size for fixed block mode in 1024 byte units
  bb is the write threshold in 1024 byte units
  dd is the maximum number of scatter/gather segments
 IOCTLS
 The tape is positioned and the drive parameters are set with ioctls
 defined in mtio.h The tape control program 'mt' uses these ioctls. Try
 to find an mt that supports all of the Linux SCSI tape ioctls and
 opens the device for writing if the tape contents will be modified
 (look for a package mt-st* from the Linux ftp sites; the GNU mt does
 not open for writing for, e.g., erase).
 The supported ioctls are:
 The following use the structure mtop:
 MTFSF   Space forward over count filemarks. Tape positioned after filemark.
 MTFSFM  As above but tape positioned before filemark.
 MTBSF	Space backward over count filemarks. Tape positioned before
        filemark.
 MTBSFM  As above but ape positioned after filemark.
 MTFSR   Space forward over count records.
 MTBSR   Space backward over count records.
 MTFSS   Space forward over count setmarks.
 MTBSS   Space backward over count setmarks.
 MTWEOF  Write count filemarks.
 MTWSM   Write count setmarks.
 MTREW   Rewind tape.
 MTOFFL  Set device off line (often rewind plus eject).
 MTNOP   Do nothing except flush the buffers.
 MTRETEN Re-tension tape.
 MTEOM   Space to end of recorded data.
 MTERASE Erase tape. If the argument is zero, the short erase command
 	is used. The long erase command is used with all other values
 	of the argument.
 MTSEEK	Seek to tape block count. Uses Tandberg-compatible seek (QFA)
        for SCSI-1 drives and SCSI-2 seek for SCSI-2 drives. The file and
 	block numbers in the status are not valid after a seek.
 MTSETBLK Set the drive block size. Setting to zero sets the drive into
        variable block mode (if applicable).
 MTSETDENSITY Sets the drive density code to arg. See drive
        documentation for available codes.
 MTLOCK and MTUNLOCK Explicitly lock/unlock the tape drive door.
 MTLOAD and MTUNLOAD Explicitly load and unload the tape. If the
 	command argument x is between MT_ST_HPLOADER_OFFSET + 1 and
 	MT_ST_HPLOADER_OFFSET + 6, the number x is used sent to the
 	drive with the command and it selects the tape slot to use of
 	HP C1553A changer.
 MTCOMPRESSION Sets compressing or uncompressing drive mode using the
 	SCSI mode page 15. Note that some drives other methods for
 	control of compression. Some drives (like the Exabytes) use
 	density codes for compression control. Some drives use another
 	mode page but this page has not been implemented in the
 	driver. Some drives without compression capability will accept
 	any compression mode without error.
 MTSETPART Moves the tape to the partition given by the argument at the
 	next tape operation. The block at which the tape is positioned
 	is the block where the tape was previously positioned in the
 	new active partition unless the next tape operation is
 	MTSEEK. In this case the tape is moved directly to the block
 	specified by MTSEEK. MTSETPART is inactive unless
 	MT_ST_CAN_PARTITIONS set.
 MTMKPART Formats the tape with one partition (argument zero) or two
 	partitions (the argument gives in megabytes the size of
 	partition 1 that is physically the first partition of the
 	tape). The drive has to support partitions with size specified
 	by the initiator. Inactive unless MT_ST_CAN_PARTITIONS set.
 MTSETDRVBUFFER
 	Is used for several purposes. The command is obtained from count
        with mask MT_SET_OPTIONS, the low order bits are used as argument.
 	This command is only allowed for the superuser (root). The
 	subcommands are:
 	0
           The drive buffer option is set to the argument. Zero means
           no buffering.
        MT_ST_BOOLEANS
           Sets the buffering options. The bits are the new states
           (enabled/disabled) the following options (in the
 	   parenthesis is specified whether the option is global or
 	   can be specified differently for each mode):
 	     MT_ST_BUFFER_WRITES write buffering (mode)
 	     MT_ST_ASYNC_WRITES asynchronous writes (mode)
             MT_ST_READ_AHEAD  read ahead (mode)
             MT_ST_TWO_FM writing of two filemarks (global)
 	     MT_ST_FAST_EOM using the SCSI spacing to EOD (global)
 	     MT_ST_AUTO_LOCK automatic locking of the drive door (global)
             MT_ST_DEF_WRITES the defaults are meant only for writes (mode)
 	     MT_ST_CAN_BSR backspacing over more than one records can
 		be used for repositioning the tape (global)
 	     MT_ST_NO_BLKLIMS the driver does not ask the block limits
 		from the drive (block size can be changed only to
 		variable) (global)
 	     MT_ST_CAN_PARTITIONS enables support for partitioned
 		tapes (global)
 	     MT_ST_SCSI2LOGICAL the logical block number is used in
 		the MTSEEK and MTIOCPOS for SCSI-2 drives instead of
 		the device dependent address. It is recommended to set
 		this flag unless there are tapes using the device
 		dependent (from the old times) (global)
 	     MT_ST_SYSV sets the SYSV semantics (mode)
 	     MT_ST_NOWAIT enables immediate mode (i.e., don't wait for
 	        the command to finish) for some commands (e.g., rewind)
 	     MT_ST_DEBUGGING debugging (global; debugging must be
 		compiled into the driver)
 	MT_ST_SETBOOLEANS
 	MT_ST_CLEARBOOLEANS
 	   Sets or clears the option bits.
        MT_ST_WRITE_THRESHOLD
           Sets the write threshold for this device to kilobytes
           specified by the lowest bits.
 	MT_ST_DEF_BLKSIZE
 	   Defines the default block size set automatically. Value
 	   0xffffff means that the default is not used any more.
 	MT_ST_DEF_DENSITY
 	MT_ST_DEF_DRVBUFFER
 	   Used to set or clear the density (8 bits), and drive buffer
 	   state (3 bits). If the value is MT_ST_CLEAR_DEFAULT
 	   (0xfffff) the default will not be used any more. Otherwise
 	   the lowermost bits of the value contain the new value of
 	   the parameter.
 	MT_ST_DEF_COMPRESSION
 	   The compression default will not be used if the value of
 	   the lowermost byte is 0xff. Otherwise the lowermost bit
 	   contains the new default. If the bits 8-15 are set to a
 	   non-zero number, and this number is not 0xff, the number is
 	   used as the compression algorithm. The value
 	   MT_ST_CLEAR_DEFAULT can be used to clear the compression
 	   default.
 	MT_ST_SET_TIMEOUT
 	   Set the normal timeout in seconds for this device. The
 	   default is 900 seconds (15 minutes). The timeout should be
 	   long enough for the retries done by the device while
 	   reading/writing.
 	MT_ST_SET_LONG_TIMEOUT
 	   Set the long timeout that is used for operations that are
 	   known to take a long time. The default is 14000 seconds
 	   (3.9 hours). For erase this value is further multiplied by
 	   eight.
 	MT_ST_SET_CLN
 	   Set the cleaning request interpretation parameters using
 	   the lowest 24 bits of the argument. The driver can set the
 	   generic status bit GMT_CLN if a cleaning request bit pattern
 	   is found from the extended sense data. Many drives set one or
 	   more bits in the extended sense data when the drive needs
 	   cleaning. The bits are device-dependent. The driver is
 	   given the number of the sense data byte (the lowest eight
 	   bits of the argument; must be >= 18 (values 1 - 17
 	   reserved) and <= the maximum requested sense data sixe), 
 	   a mask to select the relevant bits (the bits 9-16), and the
 	   bit pattern (bits 17-23). If the bit pattern is zero, one
 	   or more bits under the mask indicate cleaning request. If
 	   the pattern is non-zero, the pattern must match the masked
 	   sense data byte.
 	   (The cleaning bit is set if the additional sense code and
 	   qualifier 00h 17h are seen regardless of the setting of
 	   MT_ST_SET_CLN.)
 The following ioctl uses the structure mtpos:
 MTIOCPOS Reads the current position from the drive. Uses
        Tandberg-compatible QFA for SCSI-1 drives and the SCSI-2
        command for the SCSI-2 drives.
 The following ioctl uses the structure mtget to return the status:
 MTIOCGET Returns some status information.
        The file number and block number within file are returned. The
        block is -1 when it can't be determined (e.g., after MTBSF).
        The drive type is either MTISSCSI1 or MTISSCSI2.
        The number of recovered errors since the previous status call
        is stored in the lower word of the field mt_erreg.
        The current block size and the density code are stored in the field
        mt_dsreg (shifts for the subfields are MT_ST_BLKSIZE_SHIFT and
        MT_ST_DENSITY_SHIFT).
 	The GMT_xxx status bits reflect the drive status. GMT_DR_OPEN
 	is set if there is no tape in the drive. GMT_EOD means either
 	end of recorded data or end of tape. GMT_EOT means end of tape.
 MISCELLANEOUS COMPILE OPTIONS
 The recovered write errors are considered fatal if ST_RECOVERED_WRITE_FATAL
 is defined.
 The maximum number of tape devices is determined by the define
 ST_MAX_TAPES. If more tapes are detected at driver initialization, the
 maximum is adjusted accordingly.
 Immediate return from tape positioning SCSI commands can be enabled by
 defining ST_NOWAIT. If this is defined, the user should take care that
 the next tape operation is not started before the previous one has
 finished. The drives and SCSI adapters should handle this condition
 gracefully, but some drive/adapter combinations are known to hang the
 SCSI bus in this case.
 The MTEOM command is by default implemented as spacing over 32767
 filemarks. With this method the file number in the status is
 correct. The user can request using direct spacing to EOD by setting
 ST_FAST_EOM 1 (or using the MT_ST_OPTIONS ioctl). In this case the file
 number will be invalid.
 When using read ahead or buffered writes the position within the file
 may not be correct after the file is closed (correct position may
 require backspacing over more than one record). The correct position
 within file can be obtained if ST_IN_FILE_POS is defined at compile
 time or the MT_ST_CAN_BSR bit is set for the drive with an ioctl.
 (The driver always backs over a filemark crossed by read ahead if the
 user does not request data that far.)
 DEBUGGING HINTS
 To enable debugging messages, edit st.c and #define DEBUG 1. As seen
 above, debugging can be switched off with an ioctl if debugging is
 compiled into the driver. The debugging output is not voluminous.
 If the tape seems to hang, I would be very interested to hear where
 the driver is waiting. With the command 'ps -l' you can see the state
 of the process using the tape. If the state is D, the process is
 waiting for something. The field WCHAN tells where the driver is
 waiting. If you have the current System.map in the correct place (in
 /boot for the procps I use) or have updated /etc/psdatabase (for kmem
 ps), ps writes the function name in the WCHAN field. If not, you have
 to look up the function from System.map.
 Note also that the timeouts are very long compared to most other
 drivers. This means that the Linux driver may appear hung although the
 real reason is that the tape firmware has got confused.
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
 <book>
 <?dbhtml filename="index.html">
 <!-- ****************************************************** -->
 <!-- Header  -->
 <!-- ****************************************************** -->
  <bookinfo>
    <title>Writing an ALSA Driver</title>
    <author>
      <firstname>Takashi</firstname>
      <surname>Iwai</surname>
      <affiliation>
        <address>
          <email>tiwai@suse.de</email>
        </address>
      </affiliation>
     </author>
     <date>November 17, 2005</date>
     <edition>0.3.6</edition>
    <abstract>
      <para>
        This document describes how to write an ALSA (Advanced Linux
        Sound Architecture) driver.
      </para>
    </abstract>
    <legalnotice>
    <para>
    Copyright (c) 2002-2005  Takashi Iwai <email>tiwai@suse.de</email>
    </para>
    <para>
    This document is free; you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version. 
    </para>
    <para>
    This document is distributed in the hope that it will be useful,
    but <emphasis>WITHOUT ANY WARRANTY</emphasis>; without even the
    implied warranty of <emphasis>MERCHANTABILITY or FITNESS FOR A
    PARTICULAR PURPOSE</emphasis>. See the GNU General Public License
    for more details.
    </para>
    <para>
    You should have received a copy of the GNU General Public
    License along with this program; if not, write to the Free
    Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
    MA 02111-1307 USA
    </para>
    </legalnotice>
  </bookinfo>
 <!-- ****************************************************** -->
 <!-- Preface  -->
 <!-- ****************************************************** -->
  <preface id="preface">
    <title>Preface</title>
    <para>
      This document describes how to write an
      <ulink url="http://www.alsa-project.org/"><citetitle>
      ALSA (Advanced Linux Sound Architecture)</citetitle></ulink>
      driver. The document focuses mainly on the PCI soundcard.
      In the case of other device types, the API might
      be different, too. However, at least the ALSA kernel API is
      consistent, and therefore it would be still a bit help for
      writing them.
    </para>
    <para>
    The target of this document is ones who already have enough
    skill of C language and have the basic knowledge of linux
    kernel programming.  This document doesn't explain the general
    topics of linux kernel codes and doesn't cover the detail of
    implementation of each low-level driver.  It describes only how is
    the standard way to write a PCI sound driver on ALSA.
    </para>
    <para>
      If you are already familiar with the older ALSA ver.0.5.x, you
    can check the drivers such as <filename>es1938.c</filename> or
    <filename>maestro3.c</filename> which have also almost the same
    code-base in the ALSA 0.5.x tree, so you can compare the differences.
    </para>
    <para>
      This document is still a draft version. Any feedbacks and
    corrections, please!!
    </para>
  </preface>
 <!-- ****************************************************** -->
 <!-- File Tree Structure  -->
 <!-- ****************************************************** -->
  <chapter id="file-tree">
    <title>File Tree Structure</title>
    <section id="file-tree-general">
      <title>General</title>
      <para>
        The ALSA drivers are provided in the two ways.
      </para>
      <para>
        One is the trees provided as a tarball or via cvs from the
      ALSA's ftp site, and another is the 2.6 (or later) Linux kernel
      tree. To synchronize both, the ALSA driver tree is split into
      two different trees: alsa-kernel and alsa-driver. The former
      contains purely the source codes for the Linux 2.6 (or later)
      tree. This tree is designed only for compilation on 2.6 or
      later environment. The latter, alsa-driver, contains many subtle
      files for compiling the ALSA driver on the outside of Linux
      kernel like configure script, the wrapper functions for older,
      2.2 and 2.4 kernels, to adapt the latest kernel API,
      and additional drivers which are still in development or in
      tests.  The drivers in alsa-driver tree will be moved to
      alsa-kernel (eventually 2.6 kernel tree) once when they are
      finished and confirmed to work fine.
      </para>
      <para>
        The file tree structure of ALSA driver is depicted below. Both
        alsa-kernel and alsa-driver have almost the same file
        structure, except for <quote>core</quote> directory. It's
        named as <quote>acore</quote> in alsa-driver tree. 
        <example>
          <title>ALSA File Tree Structure</title>
          <literallayout>
        sound
                /core
                        /oss
                        /seq
                                /oss
                                /instr
                /ioctl32
                /include
                /drivers
                        /mpu401
                        /opl3
                /i2c
                        /l3
                /synth
                        /emux
                /pci
                        /(cards)
                /isa
                        /(cards)
                /arm
                /ppc
                /sparc
                /usb
                /pcmcia /(cards)
                /oss
          </literallayout>
        </example>
      </para>
    </section>
    <section id="file-tree-core-directory">
      <title>core directory</title>
      <para>
        This directory contains the middle layer, that is, the heart
      of ALSA drivers. In this directory, the native ALSA modules are
      stored. The sub-directories contain different modules and are
      dependent upon the kernel config. 
      </para>
      <section id="file-tree-core-directory-oss">
        <title>core/oss</title>
        <para>
          The codes for PCM and mixer OSS emulation modules are stored
        in this directory. The rawmidi OSS emulation is included in
        the ALSA rawmidi code since it's quite small. The sequencer
        code is stored in core/seq/oss directory (see
        <link linkend="file-tree-core-directory-seq-oss"><citetitle>
        below</citetitle></link>).
        </para>
      </section>
      <section id="file-tree-core-directory-ioctl32">
        <title>core/ioctl32</title>
        <para>
          This directory contains the 32bit-ioctl wrappers for 64bit
        architectures such like x86-64, ppc64 and sparc64. For 32bit
        and alpha architectures, these are not compiled. 
        </para>
      </section>
      <section id="file-tree-core-directory-seq">
        <title>core/seq</title>
        <para>
          This and its sub-directories are for the ALSA
        sequencer. This directory contains the sequencer core and
        primary sequencer modules such like snd-seq-midi,
        snd-seq-virmidi, etc. They are compiled only when
        <constant>CONFIG_SND_SEQUENCER</constant> is set in the kernel
        config. 
        </para>
      </section>
      <section id="file-tree-core-directory-seq-oss">
        <title>core/seq/oss</title>
        <para>
          This contains the OSS sequencer emulation codes.
        </para>
      </section>
      <section id="file-tree-core-directory-deq-instr">
        <title>core/seq/instr</title>
        <para>
          This directory contains the modules for the sequencer
        instrument layer. 
        </para>
      </section>
    </section>
    <section id="file-tree-include-directory">
      <title>include directory</title>
      <para>
        This is the place for the public header files of ALSA drivers,
      which are to be exported to the user-space, or included by
      several files at different directories. Basically, the private
      header files should not be placed in this directory, but you may
      still find files there, due to historical reason :) 
      </para>
    </section>
    <section id="file-tree-drivers-directory">
      <title>drivers directory</title>
      <para>
        This directory contains the codes shared among different drivers
      on the different architectures.  They are hence supposed not to be
      architecture-specific.
      For example, the dummy pcm driver and the serial MIDI
      driver are found in this directory. In the sub-directories,
      there are the codes for components which are independent from
      bus and cpu architectures. 
      </para>
      <section id="file-tree-drivers-directory-mpu401">
        <title>drivers/mpu401</title>
        <para>
          The MPU401 and MPU401-UART modules are stored here.
        </para>
      </section>
      <section id="file-tree-drivers-directory-opl3">
        <title>drivers/opl3 and opl4</title>
        <para>
          The OPL3 and OPL4 FM-synth stuff is found here.
        </para>
      </section>
    </section>
    <section id="file-tree-i2c-directory">
      <title>i2c directory</title>
      <para>
        This contains the ALSA i2c components.
      </para>
      <para>
        Although there is a standard i2c layer on Linux, ALSA has its
      own i2c codes for some cards, because the soundcard needs only a
      simple operation and the standard i2c API is too complicated for
      such a purpose. 
      </para>
      <section id="file-tree-i2c-directory-l3">
        <title>i2c/l3</title>
        <para>
          This is a sub-directory for ARM L3 i2c.
        </para>
      </section>
    </section>
    <section id="file-tree-synth-directory">
        <title>synth directory</title>
        <para>
          This contains the synth middle-level modules.
        </para>
        <para>
          So far, there is only Emu8000/Emu10k1 synth driver under
        synth/emux sub-directory. 
        </para>
    </section>
    <section id="file-tree-pci-directory">
      <title>pci directory</title>
      <para>
        This and its sub-directories hold the top-level card modules
      for PCI soundcards and the codes specific to the PCI BUS.
      </para>
      <para>
        The drivers compiled from a single file is stored directly on
      pci directory, while the drivers with several source files are
      stored on its own sub-directory (e.g. emu10k1, ice1712). 
      </para>
    </section>
    <section id="file-tree-isa-directory">
      <title>isa directory</title>
      <para>
        This and its sub-directories hold the top-level card modules
      for ISA soundcards. 
      </para>
    </section>
    <section id="file-tree-arm-ppc-sparc-directories">
      <title>arm, ppc, and sparc directories</title>
      <para>
        These are for the top-level card modules which are
      specific to each given architecture. 
      </para>
    </section>
    <section id="file-tree-usb-directory">
      <title>usb directory</title>
      <para>
        This contains the USB-audio driver. On the latest version, the
      USB MIDI driver is integrated together with usb-audio driver. 
      </para>
    </section>
    <section id="file-tree-pcmcia-directory">
      <title>pcmcia directory</title>
      <para>
        The PCMCIA, especially PCCard drivers will go here. CardBus
      drivers will be on pci directory, because its API is identical
      with the standard PCI cards. 
      </para>
    </section>
    <section id="file-tree-oss-directory">
      <title>oss directory</title>
      <para>
        The OSS/Lite source files are stored here on Linux 2.6 (or
      later) tree. (In the ALSA driver tarball, it's empty, of course :) 
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Basic Flow for PCI Drivers  -->
 <!-- ****************************************************** -->
  <chapter id="basic-flow">
    <title>Basic Flow for PCI Drivers</title>
    <section id="basic-flow-outline">
      <title>Outline</title>
      <para>
        The minimum flow of PCI soundcard is like the following:
        <itemizedlist>
          <listitem><para>define the PCI ID table (see the section
          <link linkend="pci-resource-entries"><citetitle>PCI Entries
          </citetitle></link>).</para></listitem> 
          <listitem><para>create <function>probe()</function> callback.</para></listitem>
          <listitem><para>create <function>remove()</function> callback.</para></listitem>
          <listitem><para>create pci_driver table which contains the three pointers above.</para></listitem>
          <listitem><para>create <function>init()</function> function just calling <function>pci_register_driver()</function> to register the pci_driver table defined above.</para></listitem>
          <listitem><para>create <function>exit()</function> function to call <function>pci_unregister_driver()</function> function.</para></listitem>
        </itemizedlist>
      </para>
    </section>
    <section id="basic-flow-example">
      <title>Full Code Example</title>
      <para>
        The code example is shown below. Some parts are kept
      unimplemented at this moment but will be filled in the
      succeeding sections. The numbers in comment lines of
      <function>snd_mychip_probe()</function> function are the
      markers. 
        <example>
          <title>Basic Flow for PCI Drivers Example</title>
          <programlisting>
 <![CDATA[
  #include <sound/driver.h>
  #include <linux/init.h>
  #include <linux/pci.h>
  #include <linux/slab.h>
  #include <sound/core.h>
  #include <sound/initval.h>
  /* module parameters (see "Module Parameters") */
  static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
  static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
  static int enable[SNDRV_CARDS] = SNDRV_DEFAULT_ENABLE_PNP;
  /* definition of the chip-specific record */
  struct mychip {
          struct snd_card *card;
          // rest of implementation will be in the section
          // "PCI Resource Managements"
  };
  /* chip-specific destructor
   * (see "PCI Resource Managements")
   */
  static int snd_mychip_free(struct mychip *chip)
  {
          .... // will be implemented later...
  }
  /* component-destructor
   * (see "Management of Cards and Components")
   */
  static int snd_mychip_dev_free(struct snd_device *device)
  {
          return snd_mychip_free(device->device_data);
  }
  /* chip-specific constructor
   * (see "Management of Cards and Components")
   */
  static int __devinit snd_mychip_create(struct snd_card *card,
                                         struct pci_dev *pci,
                                         struct mychip **rchip)
  {
          struct mychip *chip;
          int err;
          static struct snd_device_ops ops = {
                 .dev_free = snd_mychip_dev_free,
          };
          *rchip = NULL;
          // check PCI availability here
          // (see "PCI Resource Managements")
          ....
          /* allocate a chip-specific data with zero filled */
          chip = kzalloc(sizeof(*chip), GFP_KERNEL);
          if (chip == NULL)
                  return -ENOMEM;
          chip->card = card;
          // rest of initialization here; will be implemented
          // later, see "PCI Resource Managements"
          ....
          if ((err = snd_device_new(card, SNDRV_DEV_LOWLEVEL,
                                    chip, &ops)) < 0) {
                  snd_mychip_free(chip);
                  return err;
          }
          snd_card_set_dev(card, &pci->dev);
          *rchip = chip;
          return 0;
  }
  /* constructor -- see "Constructor" sub-section */
  static int __devinit snd_mychip_probe(struct pci_dev *pci,
                               const struct pci_device_id *pci_id)
  {
          static int dev;
          struct snd_card *card;
          struct mychip *chip;
          int err;
          /* (1) */
          if (dev >= SNDRV_CARDS)
                  return -ENODEV;
          if (!enable[dev]) {
                  dev++;
                  return -ENOENT;
          }
          /* (2) */
          card = snd_card_new(index[dev], id[dev], THIS_MODULE, 0);
          if (card == NULL)
                  return -ENOMEM;
          /* (3) */
          if ((err = snd_mychip_create(card, pci, &chip)) < 0) {
                  snd_card_free(card);
                  return err;
          }
          /* (4) */
          strcpy(card->driver, "My Chip");
          strcpy(card->shortname, "My Own Chip 123");
          sprintf(card->longname, "%s at 0x%lx irq %i",
                  card->shortname, chip->ioport, chip->irq);
          /* (5) */
          .... // implemented later
          /* (6) */
          if ((err = snd_card_register(card)) < 0) {
                  snd_card_free(card);
                  return err;
          }
          /* (7) */
          pci_set_drvdata(pci, card);
          dev++;
          return 0;
  }
  /* destructor -- see "Destructor" sub-section */
  static void __devexit snd_mychip_remove(struct pci_dev *pci)
  {
          snd_card_free(pci_get_drvdata(pci));
          pci_set_drvdata(pci, NULL);
  }
 ]]>
          </programlisting>
        </example>
      </para>
    </section>
    <section id="basic-flow-constructor">
      <title>Constructor</title>
      <para>
        The real constructor of PCI drivers is probe callback. The
      probe callback and other component-constructors which are called
      from probe callback should be defined with
      <parameter>__devinit</parameter> prefix. You 
      cannot use <parameter>__init</parameter> prefix for them,
      because any PCI device could be a hotplug device. 
      </para>
      <para>
        In the probe callback, the following scheme is often used.
      </para>
      <section id="basic-flow-constructor-device-index">
        <title>1) Check and increment the device index.</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int dev;
  ....
  if (dev >= SNDRV_CARDS)
          return -ENODEV;
  if (!enable[dev]) {
          dev++;
          return -ENOENT;
  }
 ]]>
            </programlisting>
          </informalexample>
        where enable[dev] is the module option.
        </para>
        <para>
          At each time probe callback is called, check the
        availability of the device. If not available, simply increment
        the device index and returns. dev will be incremented also
        later (<link
        linkend="basic-flow-constructor-set-pci"><citetitle>step
        7</citetitle></link>). 
        </para>
      </section>
      <section id="basic-flow-constructor-create-card">
        <title>2) Create a card instance</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  struct snd_card *card;
  ....
  card = snd_card_new(index[dev], id[dev], THIS_MODULE, 0);
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          The detail will be explained in the section
          <link linkend="card-management-card-instance"><citetitle>
          Management of Cards and Components</citetitle></link>.
        </para>
      </section>
      <section id="basic-flow-constructor-create-main">
        <title>3) Create a main component</title>
        <para>
          In this part, the PCI resources are allocated.
          <informalexample>
            <programlisting>
 <![CDATA[
  struct mychip *chip;
  ....
  if ((err = snd_mychip_create(card, pci, &chip)) < 0) {
          snd_card_free(card);
          return err;
  }
 ]]>
            </programlisting>
          </informalexample>
          The detail will be explained in the section <link
        linkend="pci-resource"><citetitle>PCI Resource
        Managements</citetitle></link>.
        </para>
      </section>
      <section id="basic-flow-constructor-main-component">
        <title>4) Set the driver ID and name strings.</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  strcpy(card->driver, "My Chip");
  strcpy(card->shortname, "My Own Chip 123");
  sprintf(card->longname, "%s at 0x%lx irq %i",
          card->shortname, chip->ioport, chip->irq);
 ]]>
            </programlisting>
          </informalexample>
          The driver field holds the minimal ID string of the
        chip. This is referred by alsa-lib's configurator, so keep it
        simple but unique. 
          Even the same driver can have different driver IDs to
        distinguish the functionality of each chip type. 
        </para>
        <para>
          The shortname field is a string shown as more verbose
        name. The longname field contains the information which is
        shown in <filename>/proc/asound/cards</filename>. 
        </para>
      </section>
      <section id="basic-flow-constructor-create-other">
        <title>5) Create other components, such as mixer, MIDI, etc.</title>
        <para>
          Here you define the basic components such as
          <link linkend="pcm-interface"><citetitle>PCM</citetitle></link>,
          mixer (e.g. <link linkend="api-ac97"><citetitle>AC97</citetitle></link>),
          MIDI (e.g. <link linkend="midi-interface"><citetitle>MPU-401</citetitle></link>),
          and other interfaces.
          Also, if you want a <link linkend="proc-interface"><citetitle>proc
        file</citetitle></link>, define it here, too.
        </para>
      </section>
      <section id="basic-flow-constructor-register-card">
        <title>6) Register the card instance.</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  if ((err = snd_card_register(card)) < 0) {
          snd_card_free(card);
          return err;
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          Will be explained in the section <link
        linkend="card-management-registration"><citetitle>Management
        of Cards and Components</citetitle></link>, too. 
        </para>
      </section>
      <section id="basic-flow-constructor-set-pci">
        <title>7) Set the PCI driver data and return zero.</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
        pci_set_drvdata(pci, card);
        dev++;
        return 0;
 ]]>
            </programlisting>
          </informalexample>
          In the above, the card record is stored. This pointer is
        referred in the remove callback and power-management
        callbacks, too. 
        </para>
      </section>
    </section>
    <section id="basic-flow-destructor">
      <title>Destructor</title>
      <para>
        The destructor, remove callback, simply releases the card
      instance. Then the ALSA middle layer will release all the
      attached components automatically. 
      </para>
      <para>
        It would be typically like the following:
        <informalexample>
          <programlisting>
 <![CDATA[
  static void __devexit snd_mychip_remove(struct pci_dev *pci)
  {
          snd_card_free(pci_get_drvdata(pci));
          pci_set_drvdata(pci, NULL);
  }
 ]]>
          </programlisting>
        </informalexample>
        The above code assumes that the card pointer is set to the PCI
 	driver data.
      </para>
    </section>
    <section id="basic-flow-header-files">
      <title>Header Files</title>
      <para>
        For the above example, at least the following include files
      are necessary. 
        <informalexample>
          <programlisting>
 <![CDATA[
  #include <sound/driver.h>
  #include <linux/init.h>
  #include <linux/pci.h>
  #include <linux/slab.h>
  #include <sound/core.h>
  #include <sound/initval.h>
 ]]>
          </programlisting>
        </informalexample>
 	where the last one is necessary only when module options are
      defined in the source file.  If the codes are split to several
      files, the file without module options don't need them.
      </para>
      <para>
        In addition to them, you'll need
      <filename>&lt;linux/interrupt.h&gt;</filename> for the interrupt
      handling, and <filename>&lt;asm/io.h&gt;</filename> for the i/o
      access. If you use <function>mdelay()</function> or
      <function>udelay()</function> functions, you'll need to include
      <filename>&lt;linux/delay.h&gt;</filename>, too. 
      </para>
      <para>
      The ALSA interfaces like PCM or control API are defined in other
      header files as <filename>&lt;sound/xxx.h&gt;</filename>.
      They have to be included after
      <filename>&lt;sound/core.h&gt;</filename>.
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Management of Cards and Components  -->
 <!-- ****************************************************** -->
  <chapter id="card-management">
    <title>Management of Cards and Components</title>
    <section id="card-management-card-instance">
      <title>Card Instance</title>
      <para>
      For each soundcard, a <quote>card</quote> record must be allocated.
      </para>
      <para>
      A card record is the headquarters of the soundcard.  It manages
      the list of whole devices (components) on the soundcard, such as
      PCM, mixers, MIDI, synthesizer, and so on.  Also, the card
      record holds the ID and the name strings of the card, manages
      the root of proc files, and controls the power-management states
      and hotplug disconnections.  The component list on the card
      record is used to manage the proper releases of resources at
      destruction. 
      </para>
      <para>
        As mentioned above, to create a card instance, call
      <function>snd_card_new()</function>.
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_card *card;
  card = snd_card_new(index, id, module, extra_size);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The function takes four arguments, the card-index number, the
        id string, the module pointer (usually
        <constant>THIS_MODULE</constant>),
        and the size of extra-data space.  The last argument is used to
        allocate card-&gt;private_data for the
        chip-specific data.  Note that this data
        <emphasis>is</emphasis> allocated by
        <function>snd_card_new()</function>.
      </para>
    </section>
    <section id="card-management-component">
      <title>Components</title>
      <para>
        After the card is created, you can attach the components
      (devices) to the card instance. On ALSA driver, a component is
      represented as a struct <structname>snd_device</structname> object.
      A component can be a PCM instance, a control interface, a raw
      MIDI interface, etc.  Each of such instances has one component
      entry.
      </para>
      <para>
        A component can be created via
        <function>snd_device_new()</function> function. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_device_new(card, SNDRV_DEV_XXX, chip, &ops);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        This takes the card pointer, the device-level
      (<constant>SNDRV_DEV_XXX</constant>), the data pointer, and the
      callback pointers (<parameter>&amp;ops</parameter>). The
      device-level defines the type of components and the order of
      registration and de-registration.  For most of components, the
      device-level is already defined.  For a user-defined component,
      you can use <constant>SNDRV_DEV_LOWLEVEL</constant>.
      </para>
      <para>
      This function itself doesn't allocate the data space. The data
      must be allocated manually beforehand, and its pointer is passed
      as the argument. This pointer is used as the identifier
      (<parameter>chip</parameter> in the above example) for the
      instance. 
      </para>
      <para>
        Each ALSA pre-defined component such as ac97 or pcm calls
      <function>snd_device_new()</function> inside its
      constructor. The destructor for each component is defined in the
      callback pointers.  Hence, you don't need to take care of
      calling a destructor for such a component.
      </para>
      <para>
        If you would like to create your own component, you need to
      set the destructor function to dev_free callback in
      <parameter>ops</parameter>, so that it can be released
      automatically via <function>snd_card_free()</function>. The
      example will be shown later as an implementation of a
      chip-specific data. 
      </para>
    </section>
    <section id="card-management-chip-specific">
      <title>Chip-Specific Data</title>
      <para>
      The chip-specific information, e.g. the i/o port address, its
      resource pointer, or the irq number, is stored in the
      chip-specific record.
        <informalexample>
          <programlisting>
 <![CDATA[
  struct mychip {
          ....
  };
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        In general, there are two ways to allocate the chip record.
      </para>
      <section id="card-management-chip-specific-snd-card-new">
        <title>1. Allocating via <function>snd_card_new()</function>.</title>
        <para>
          As mentioned above, you can pass the extra-data-length to the 4th argument of <function>snd_card_new()</function>, i.e.
          <informalexample>
            <programlisting>
 <![CDATA[
  card = snd_card_new(index[dev], id[dev], THIS_MODULE, sizeof(struct mychip));
 ]]>
            </programlisting>
          </informalexample>
          whether struct <structname>mychip</structname> is the type of the chip record.
        </para>
        <para>
          In return, the allocated record can be accessed as
          <informalexample>
            <programlisting>
 <![CDATA[
  struct mychip *chip = (struct mychip *)card->private_data;
 ]]>
            </programlisting>
          </informalexample>
          With this method, you don't have to allocate twice.
          The record is released together with the card instance.
        </para>
      </section>
      <section id="card-management-chip-specific-allocate-extra">
        <title>2. Allocating an extra device.</title>
        <para>
          After allocating a card instance via
          <function>snd_card_new()</function> (with
          <constant>NULL</constant> on the 4th arg), call
          <function>kzalloc()</function>. 
          <informalexample>
            <programlisting>
 <![CDATA[
  struct snd_card *card;
  struct mychip *chip;
  card = snd_card_new(index[dev], id[dev], THIS_MODULE, NULL);
  .....
  chip = kzalloc(sizeof(*chip), GFP_KERNEL);
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          The chip record should have the field to hold the card
          pointer at least, 
          <informalexample>
            <programlisting>
 <![CDATA[
  struct mychip {
          struct snd_card *card;
          ....
  };
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          Then, set the card pointer in the returned chip instance.
          <informalexample>
            <programlisting>
 <![CDATA[
  chip->card = card;
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          Next, initialize the fields, and register this chip
          record as a low-level device with a specified
          <parameter>ops</parameter>, 
          <informalexample>
            <programlisting>
 <![CDATA[
  static struct snd_device_ops ops = {
          .dev_free =        snd_mychip_dev_free,
  };
  ....
  snd_device_new(card, SNDRV_DEV_LOWLEVEL, chip, &ops);
 ]]>
            </programlisting>
          </informalexample>
          <function>snd_mychip_dev_free()</function> is the
        device-destructor function, which will call the real
        destructor. 
        </para>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_mychip_dev_free(struct snd_device *device)
  {
          return snd_mychip_free(device->device_data);
  }
 ]]>
            </programlisting>
          </informalexample>
          where <function>snd_mychip_free()</function> is the real destructor.
        </para>
      </section>
    </section>
    <section id="card-management-registration">
      <title>Registration and Release</title>
      <para>
        After all components are assigned, register the card instance
      by calling <function>snd_card_register()</function>. The access
      to the device files are enabled at this point. That is, before
      <function>snd_card_register()</function> is called, the
      components are safely inaccessible from external side. If this
      call fails, exit the probe function after releasing the card via
      <function>snd_card_free()</function>. 
      </para>
      <para>
        For releasing the card instance, you can call simply
      <function>snd_card_free()</function>. As already mentioned, all
      components are released automatically by this call. 
      </para>
      <para>
        As further notes, the destructors (both
      <function>snd_mychip_dev_free</function> and
      <function>snd_mychip_free</function>) cannot be defined with
      <parameter>__devexit</parameter> prefix, because they may be
      called from the constructor, too, at the false path. 
      </para>
      <para>
      For a device which allows hotplugging, you can use
      <function>snd_card_free_when_closed</function>.  This one will
      postpone the destruction until all devices are closed.
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- PCI Resource Managements  -->
 <!-- ****************************************************** -->
  <chapter id="pci-resource">
    <title>PCI Resource Managements</title>
    <section id="pci-resource-example">
      <title>Full Code Example</title>
      <para>
        In this section, we'll finish the chip-specific constructor,
      destructor and PCI entries. The example code is shown first,
      below. 
        <example>
          <title>PCI Resource Managements Example</title>
          <programlisting>
 <![CDATA[
  struct mychip {
          struct snd_card *card;
          struct pci_dev *pci;
          unsigned long port;
          int irq;
  };
  static int snd_mychip_free(struct mychip *chip)
  {
          /* disable hardware here if any */
          .... // (not implemented in this document)
          /* release the irq */
          if (chip->irq >= 0)
                  free_irq(chip->irq, (void *)chip);
          /* release the i/o ports & memory */
          pci_release_regions(chip->pci);
          /* disable the PCI entry */
          pci_disable_device(chip->pci);
          /* release the data */
          kfree(chip);
          return 0;
  }
  /* chip-specific constructor */
  static int __devinit snd_mychip_create(struct snd_card *card,
                                         struct pci_dev *pci,
                                         struct mychip **rchip)
  {
          struct mychip *chip;
          int err;
          static struct snd_device_ops ops = {
                 .dev_free = snd_mychip_dev_free,
          };
          *rchip = NULL;
          /* initialize the PCI entry */
          if ((err = pci_enable_device(pci)) < 0)
                  return err;
          /* check PCI availability (28bit DMA) */
          if (pci_set_dma_mask(pci, DMA_28BIT_MASK) < 0 ||
              pci_set_consistent_dma_mask(pci, DMA_28BIT_MASK) < 0) {
                  printk(KERN_ERR "error to set 28bit mask DMA\n");
                  pci_disable_device(pci);
                  return -ENXIO;
          }
          chip = kzalloc(sizeof(*chip), GFP_KERNEL);
          if (chip == NULL) {
                  pci_disable_device(pci);
                  return -ENOMEM;
          }
          /* initialize the stuff */
          chip->card = card;
          chip->pci = pci;
          chip->irq = -1;
          /* (1) PCI resource allocation */
          if ((err = pci_request_regions(pci, "My Chip")) < 0) {
                  kfree(chip);
                  pci_disable_device(pci);
                  return err;
          }
          chip->port = pci_resource_start(pci, 0);
          if (request_irq(pci->irq, snd_mychip_interrupt,
                          IRQF_DISABLED|IRQF_SHARED, "My Chip", chip)) {
                  printk(KERN_ERR "cannot grab irq %d\n", pci->irq);
                  snd_mychip_free(chip);
                  return -EBUSY;
          }
          chip->irq = pci->irq;
          /* (2) initialization of the chip hardware */
          .... //   (not implemented in this document)
          if ((err = snd_device_new(card, SNDRV_DEV_LOWLEVEL,
                                    chip, &ops)) < 0) {
                  snd_mychip_free(chip);
                  return err;
          }
          snd_card_set_dev(card, &pci->dev);
          *rchip = chip;
          return 0;
  }        
  /* PCI IDs */
  static struct pci_device_id snd_mychip_ids[] = {
          { PCI_VENDOR_ID_FOO, PCI_DEVICE_ID_BAR,
            PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0, },
          ....
          { 0, }
  };
  MODULE_DEVICE_TABLE(pci, snd_mychip_ids);
  /* pci_driver definition */
  static struct pci_driver driver = {
          .name = "My Own Chip",
          .id_table = snd_mychip_ids,
          .probe = snd_mychip_probe,
          .remove = __devexit_p(snd_mychip_remove),
  };
  /* initialization of the module */
  static int __init alsa_card_mychip_init(void)
  {
          return pci_register_driver(&driver);
  }
  /* clean up the module */
  static void __exit alsa_card_mychip_exit(void)
  {
          pci_unregister_driver(&driver);
  }
  module_init(alsa_card_mychip_init)
  module_exit(alsa_card_mychip_exit)
  EXPORT_NO_SYMBOLS; /* for old kernels only */
 ]]>
          </programlisting>
        </example>
      </para>
    </section>
    <section id="pci-resource-some-haftas">
      <title>Some Hafta's</title>
      <para>
        The allocation of PCI resources is done in the
      <function>probe()</function> function, and usually an extra
      <function>xxx_create()</function> function is written for this
      purpose.
      </para>
      <para>
        In the case of PCI devices, you have to call at first
      <function>pci_enable_device()</function> function before
      allocating resources. Also, you need to set the proper PCI DMA
      mask to limit the accessed i/o range. In some cases, you might
      need to call <function>pci_set_master()</function> function,
      too.
      </para>
      <para>
        Suppose the 28bit mask, and the code to be added would be like:
        <informalexample>
          <programlisting>
 <![CDATA[
  if ((err = pci_enable_device(pci)) < 0)
          return err;
  if (pci_set_dma_mask(pci, DMA_28BIT_MASK) < 0 ||
      pci_set_consistent_dma_mask(pci, DMA_28BIT_MASK) < 0) {
          printk(KERN_ERR "error to set 28bit mask DMA\n");
          pci_disable_device(pci);
          return -ENXIO;
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
    <section id="pci-resource-resource-allocation">
      <title>Resource Allocation</title>
      <para>
        The allocation of I/O ports and irqs are done via standard kernel
      functions. Unlike ALSA ver.0.5.x., there are no helpers for
      that. And these resources must be released in the destructor
      function (see below). Also, on ALSA 0.9.x, you don't need to
      allocate (pseudo-)DMA for PCI like ALSA 0.5.x.
      </para>
      <para>
        Now assume that this PCI device has an I/O port with 8 bytes
        and an interrupt. Then struct <structname>mychip</structname> will have the
        following fields:
        <informalexample>
          <programlisting>
 <![CDATA[
  struct mychip {
          struct snd_card *card;
          unsigned long port;
          int irq;
  };
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        For an i/o port (and also a memory region), you need to have
      the resource pointer for the standard resource management. For
      an irq, you have to keep only the irq number (integer). But you
      need to initialize this number as -1 before actual allocation,
      since irq 0 is valid. The port address and its resource pointer
      can be initialized as null by
      <function>kzalloc()</function> automatically, so you
      don't have to take care of resetting them. 
      </para>
      <para>
        The allocation of an i/o port is done like this:
        <informalexample>
          <programlisting>
 <![CDATA[
  if ((err = pci_request_regions(pci, "My Chip")) < 0) { 
          kfree(chip);
          pci_disable_device(pci);
          return err;
  }
  chip->port = pci_resource_start(pci, 0);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        <!-- obsolete -->
        It will reserve the i/o port region of 8 bytes of the given
      PCI device. The returned value, chip-&gt;res_port, is allocated
      via <function>kmalloc()</function> by
      <function>request_region()</function>. The pointer must be
      released via <function>kfree()</function>, but there is some
      problem regarding this. This issue will be explained more below.
      </para>
      <para>
        The allocation of an interrupt source is done like this:
        <informalexample>
          <programlisting>
 <![CDATA[
  if (request_irq(pci->irq, snd_mychip_interrupt,
                  IRQF_DISABLED|IRQF_SHARED, "My Chip", chip)) {
          printk(KERN_ERR "cannot grab irq %d\n", pci->irq);
          snd_mychip_free(chip);
          return -EBUSY;
  }
  chip->irq = pci->irq;
 ]]>
          </programlisting>
        </informalexample>
        where <function>snd_mychip_interrupt()</function> is the
      interrupt handler defined <link
      linkend="pcm-interface-interrupt-handler"><citetitle>later</citetitle></link>.
      Note that chip-&gt;irq should be defined
      only when <function>request_irq()</function> succeeded.
      </para>
      <para>
      On the PCI bus, the interrupts can be shared. Thus,
      <constant>IRQF_SHARED</constant> is given as the interrupt flag of
      <function>request_irq()</function>. 
      </para>
      <para>
        The last argument of <function>request_irq()</function> is the
      data pointer passed to the interrupt handler. Usually, the
      chip-specific record is used for that, but you can use what you
      like, too. 
      </para>
      <para>
        I won't define the detail of the interrupt handler at this
        point, but at least its appearance can be explained now. The
        interrupt handler looks usually like the following: 
        <informalexample>
          <programlisting>
 <![CDATA[
  static irqreturn_t snd_mychip_interrupt(int irq, void *dev_id,
                                          struct pt_regs *regs)
  {
          struct mychip *chip = dev_id;
          ....
          return IRQ_HANDLED;
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        Now let's write the corresponding destructor for the resources
      above. The role of destructor is simple: disable the hardware
      (if already activated) and release the resources. So far, we
      have no hardware part, so the disabling is not written here. 
      </para>
      <para>
        For releasing the resources, <quote>check-and-release</quote>
        method is a safer way. For the interrupt, do like this: 
        <informalexample>
          <programlisting>
 <![CDATA[
  if (chip->irq >= 0)
          free_irq(chip->irq, (void *)chip);
 ]]>
          </programlisting>
        </informalexample>
        Since the irq number can start from 0, you should initialize
        chip-&gt;irq with a negative value (e.g. -1), so that you can
        check the validity of the irq number as above.
      </para>
      <para>
        When you requested I/O ports or memory regions via
 	<function>pci_request_region()</function> or
 	<function>pci_request_regions()</function> like this example,
 	release the resource(s) using the corresponding function,
 	<function>pci_release_region()</function> or
 	<function>pci_release_regions()</function>.
        <informalexample>
          <programlisting>
 <![CDATA[
  pci_release_regions(chip->pci);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
 	When you requested manually via <function>request_region()</function>
 	or <function>request_mem_region</function>, you can release it via
 	<function>release_resource()</function>.  Suppose that you keep
 	the resource pointer returned from <function>request_region()</function>
 	in chip-&gt;res_port, the release procedure looks like below:
        <informalexample>
          <programlisting>
 <![CDATA[
  release_and_free_resource(chip->res_port);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
      Don't forget to call <function>pci_disable_device()</function>
      before all finished.
      </para>
      <para>
        And finally, release the chip-specific record.
        <informalexample>
          <programlisting>
 <![CDATA[
  kfree(chip);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
      Again, remember that you cannot
      set <parameter>__devexit</parameter> prefix for this destructor. 
      </para>
      <para>
      We didn't implement the hardware-disabling part in the above.
      If you need to do this, please note that the destructor may be
      called even before the initialization of the chip is completed.
      It would be better to have a flag to skip the hardware-disabling
      if the hardware was not initialized yet.
      </para>
      <para>
      When the chip-data is assigned to the card using
      <function>snd_device_new()</function> with
      <constant>SNDRV_DEV_LOWLELVEL</constant> , its destructor is 
      called at the last.  That is, it is assured that all other
      components like PCMs and controls have been already released.
      You don't have to call stopping PCMs, etc. explicitly, but just
      stop the hardware in the low-level.
      </para>
      <para>
        The management of a memory-mapped region is almost as same as
        the management of an i/o port. You'll need three fields like
        the following: 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct mychip {
          ....
          unsigned long iobase_phys;
          void __iomem *iobase_virt;
  };
 ]]>
          </programlisting>
        </informalexample>
        and the allocation would be like below:
        <informalexample>
          <programlisting>
 <![CDATA[
  if ((err = pci_request_regions(pci, "My Chip")) < 0) {
          kfree(chip);
          return err;
  }
  chip->iobase_phys = pci_resource_start(pci, 0);
  chip->iobase_virt = ioremap_nocache(chip->iobase_phys,
                                      pci_resource_len(pci, 0));
 ]]>
          </programlisting>
        </informalexample>
        and the corresponding destructor would be:
        <informalexample>
          <programlisting>
 <![CDATA[
  static int snd_mychip_free(struct mychip *chip)
  {
          ....
          if (chip->iobase_virt)
                  iounmap(chip->iobase_virt);
          ....
          pci_release_regions(chip->pci);
          ....
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
    <section id="pci-resource-device-struct">
      <title>Registration of Device Struct</title>
      <para>
 	At some point, typically after calling <function>snd_device_new()</function>,
 	you need to register the struct <structname>device</structname> of the chip
 	you're handling for udev and co.  ALSA provides a macro for compatibility with
 	older kernels.  Simply call like the following:
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_card_set_dev(card, &pci->dev);
 ]]>
          </programlisting>
        </informalexample>
 	so that it stores the PCI's device pointer to the card.  This will be
 	referred by ALSA core functions later when the devices are registered.
      </para>
      <para>
 	In the case of non-PCI, pass the proper device struct pointer of the BUS
 	instead.  (In the case of legacy ISA without PnP, you don't have to do
 	anything.)
      </para>
    </section>
    <section id="pci-resource-entries">
      <title>PCI Entries</title>
      <para>
        So far, so good. Let's finish the rest of missing PCI
      stuffs. At first, we need a
      <structname>pci_device_id</structname> table for this
      chipset. It's a table of PCI vendor/device ID number, and some
      masks. 
      </para>
      <para>
        For example,
        <informalexample>
          <programlisting>
 <![CDATA[
  static struct pci_device_id snd_mychip_ids[] = {
          { PCI_VENDOR_ID_FOO, PCI_DEVICE_ID_BAR,
            PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0, },
          ....
          { 0, }
  };
  MODULE_DEVICE_TABLE(pci, snd_mychip_ids);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The first and second fields of
      <structname>pci_device_id</structname> struct are the vendor and
      device IDs. If you have nothing special to filter the matching
      devices, you can use the rest of fields like above. The last
      field of <structname>pci_device_id</structname> struct is a
      private data for this entry. You can specify any value here, for
      example, to tell the type of different operations per each
      device IDs. Such an example is found in intel8x0 driver. 
      </para>
      <para>
        The last entry of this list is the terminator. You must
      specify this all-zero entry. 
      </para>
      <para>
        Then, prepare the <structname>pci_driver</structname> record:
        <informalexample>
          <programlisting>
 <![CDATA[
  static struct pci_driver driver = {
          .name = "My Own Chip",
          .id_table = snd_mychip_ids,
          .probe = snd_mychip_probe,
          .remove = __devexit_p(snd_mychip_remove),
  };
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The <structfield>probe</structfield> and
      <structfield>remove</structfield> functions are what we already
      defined in 
      the previous sections. The <structfield>remove</structfield> should
      be defined with 
      <function>__devexit_p()</function> macro, so that it's not
      defined for built-in (and non-hot-pluggable) case. The
      <structfield>name</structfield> 
      field is the name string of this device. Note that you must not
      use a slash <quote>/</quote> in this string. 
      </para>
      <para>
        And at last, the module entries:
        <informalexample>
          <programlisting>
 <![CDATA[
  static int __init alsa_card_mychip_init(void)
  {
          return pci_register_driver(&driver);
  }
  static void __exit alsa_card_mychip_exit(void)
  {
          pci_unregister_driver(&driver);
  }
  module_init(alsa_card_mychip_init)
  module_exit(alsa_card_mychip_exit)
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        Note that these module entries are tagged with
      <parameter>__init</parameter> and 
      <parameter>__exit</parameter> prefixes, not
      <parameter>__devinit</parameter> nor
      <parameter>__devexit</parameter>.
      </para>
      <para>
        Oh, one thing was forgotten. If you have no exported symbols,
        you need to declare it on 2.2 or 2.4 kernels (on 2.6 kernels
        it's not necessary, though).
        <informalexample>
          <programlisting>
 <![CDATA[
  EXPORT_NO_SYMBOLS;
 ]]>
          </programlisting>
        </informalexample>
        That's all!
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- PCM Interface  -->
 <!-- ****************************************************** -->
  <chapter id="pcm-interface">
    <title>PCM Interface</title>
    <section id="pcm-interface-general">
      <title>General</title>
      <para>
        The PCM middle layer of ALSA is quite powerful and it is only
      necessary for each driver to implement the low-level functions
      to access its hardware.
      </para>
      <para>
        For accessing to the PCM layer, you need to include
      <filename>&lt;sound/pcm.h&gt;</filename> above all. In addition,
      <filename>&lt;sound/pcm_params.h&gt;</filename> might be needed
      if you access to some functions related with hw_param. 
      </para>
      <para>
        Each card device can have up to four pcm instances. A pcm
      instance corresponds to a pcm device file. The limitation of
      number of instances comes only from the available bit size of
      the linux's device number. Once when 64bit device number is
      used, we'll have more available pcm instances. 
      </para>
      <para>
        A pcm instance consists of pcm playback and capture streams,
      and each pcm stream consists of one or more pcm substreams. Some
      soundcard supports the multiple-playback function. For example,
      emu10k1 has a PCM playback of 32 stereo substreams. In this case, at
      each open, a free substream is (usually) automatically chosen
      and opened. Meanwhile, when only one substream exists and it was
      already opened, the succeeding open will result in the blocking
      or the error with <constant>EAGAIN</constant> according to the
      file open mode. But you don't have to know the detail in your
      driver. The PCM middle layer will take all such jobs. 
      </para>
    </section>
    <section id="pcm-interface-example">
      <title>Full Code Example</title>
      <para>
      The example code below does not include any hardware access
      routines but shows only the skeleton, how to build up the PCM
      interfaces.
        <example>
          <title>PCM Example Code</title>
          <programlisting>
 <![CDATA[
  #include <sound/pcm.h>
  ....
  /* hardware definition */
  static struct snd_pcm_hardware snd_mychip_playback_hw = {
          .info = (SNDRV_PCM_INFO_MMAP |
                   SNDRV_PCM_INFO_INTERLEAVED |
                   SNDRV_PCM_INFO_BLOCK_TRANSFER |
                   SNDRV_PCM_INFO_MMAP_VALID),
          .formats =          SNDRV_PCM_FMTBIT_S16_LE,
          .rates =            SNDRV_PCM_RATE_8000_48000,
          .rate_min =         8000,
          .rate_max =         48000,
          .channels_min =     2,
          .channels_max =     2,
          .buffer_bytes_max = 32768,
          .period_bytes_min = 4096,
          .period_bytes_max = 32768,
          .periods_min =      1,
          .periods_max =      1024,
  };
  /* hardware definition */
  static struct snd_pcm_hardware snd_mychip_capture_hw = {
          .info = (SNDRV_PCM_INFO_MMAP |
                   SNDRV_PCM_INFO_INTERLEAVED |
                   SNDRV_PCM_INFO_BLOCK_TRANSFER |
                   SNDRV_PCM_INFO_MMAP_VALID),
          .formats =          SNDRV_PCM_FMTBIT_S16_LE,
          .rates =            SNDRV_PCM_RATE_8000_48000,
          .rate_min =         8000,
          .rate_max =         48000,
          .channels_min =     2,
          .channels_max =     2,
          .buffer_bytes_max = 32768,
          .period_bytes_min = 4096,
          .period_bytes_max = 32768,
          .periods_min =      1,
          .periods_max =      1024,
  };
  /* open callback */
  static int snd_mychip_playback_open(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          struct snd_pcm_runtime *runtime = substream->runtime;
          runtime->hw = snd_mychip_playback_hw;
          // more hardware-initialization will be done here
          return 0;
  }
  /* close callback */
  static int snd_mychip_playback_close(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          // the hardware-specific codes will be here
          return 0;
  }
  /* open callback */
  static int snd_mychip_capture_open(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          struct snd_pcm_runtime *runtime = substream->runtime;
          runtime->hw = snd_mychip_capture_hw;
          // more hardware-initialization will be done here
          return 0;
  }
  /* close callback */
  static int snd_mychip_capture_close(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          // the hardware-specific codes will be here
          return 0;
  }
  /* hw_params callback */
  static int snd_mychip_pcm_hw_params(struct snd_pcm_substream *substream,
                               struct snd_pcm_hw_params *hw_params)
  {
          return snd_pcm_lib_malloc_pages(substream,
                                     params_buffer_bytes(hw_params));
  }
  /* hw_free callback */
  static int snd_mychip_pcm_hw_free(struct snd_pcm_substream *substream)
  {
          return snd_pcm_lib_free_pages(substream);
  }
  /* prepare callback */
  static int snd_mychip_pcm_prepare(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          struct snd_pcm_runtime *runtime = substream->runtime;
          /* set up the hardware with the current configuration
           * for example...
           */
          mychip_set_sample_format(chip, runtime->format);
          mychip_set_sample_rate(chip, runtime->rate);
          mychip_set_channels(chip, runtime->channels);
          mychip_set_dma_setup(chip, runtime->dma_addr,
                               chip->buffer_size,
                               chip->period_size);
          return 0;
  }
  /* trigger callback */
  static int snd_mychip_pcm_trigger(struct snd_pcm_substream *substream,
                                    int cmd)
  {
          switch (cmd) {
          case SNDRV_PCM_TRIGGER_START:
                  // do something to start the PCM engine
                  break;
          case SNDRV_PCM_TRIGGER_STOP:
                  // do something to stop the PCM engine
                  break;
          default:
                  return -EINVAL;
          }
  }
  /* pointer callback */
  static snd_pcm_uframes_t
  snd_mychip_pcm_pointer(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          unsigned int current_ptr;
          /* get the current hardware pointer */
          current_ptr = mychip_get_hw_pointer(chip);
          return current_ptr;
  }
  /* operators */
  static struct snd_pcm_ops snd_mychip_playback_ops = {
          .open =        snd_mychip_playback_open,
          .close =       snd_mychip_playback_close,
          .ioctl =       snd_pcm_lib_ioctl,
          .hw_params =   snd_mychip_pcm_hw_params,
          .hw_free =     snd_mychip_pcm_hw_free,
          .prepare =     snd_mychip_pcm_prepare,
          .trigger =     snd_mychip_pcm_trigger,
          .pointer =     snd_mychip_pcm_pointer,
  };
  /* operators */
  static struct snd_pcm_ops snd_mychip_capture_ops = {
          .open =        snd_mychip_capture_open,
          .close =       snd_mychip_capture_close,
          .ioctl =       snd_pcm_lib_ioctl,
          .hw_params =   snd_mychip_pcm_hw_params,
          .hw_free =     snd_mychip_pcm_hw_free,
          .prepare =     snd_mychip_pcm_prepare,
          .trigger =     snd_mychip_pcm_trigger,
          .pointer =     snd_mychip_pcm_pointer,
  };
  /*
   *  definitions of capture are omitted here...
   */
  /* create a pcm device */
  static int __devinit snd_mychip_new_pcm(struct mychip *chip)
  {
          struct snd_pcm *pcm;
          int err;
          if ((err = snd_pcm_new(chip->card, "My Chip", 0, 1, 1,
                                 &pcm)) < 0) 
                  return err;
          pcm->private_data = chip;
          strcpy(pcm->name, "My Chip");
          chip->pcm = pcm;
          /* set operators */
          snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK,
                          &snd_mychip_playback_ops);
          snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE,
                          &snd_mychip_capture_ops);
          /* pre-allocation of buffers */
          /* NOTE: this may fail */
          snd_pcm_lib_preallocate_pages_for_all(pcm, SNDRV_DMA_TYPE_DEV,
                                                snd_dma_pci_data(chip->pci),
                                                64*1024, 64*1024);
          return 0;
  }
 ]]>
          </programlisting>
        </example>
      </para>
    </section>
    <section id="pcm-interface-constructor">
      <title>Constructor</title>
      <para>
        A pcm instance is allocated by <function>snd_pcm_new()</function>
      function. It would be better to create a constructor for pcm,
      namely, 
        <informalexample>
          <programlisting>
 <![CDATA[
  static int __devinit snd_mychip_new_pcm(struct mychip *chip)
  {
          struct snd_pcm *pcm;
          int err;
          if ((err = snd_pcm_new(chip->card, "My Chip", 0, 1, 1,
                                 &pcm)) < 0) 
                  return err;
          pcm->private_data = chip;
          strcpy(pcm->name, "My Chip");
          chip->pcm = pcm;
 	  ....
          return 0;
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The <function>snd_pcm_new()</function> function takes the four
      arguments. The first argument is the card pointer to which this
      pcm is assigned, and the second is the ID string. 
      </para>
      <para>
        The third argument (<parameter>index</parameter>, 0 in the
      above) is the index of this new pcm. It begins from zero. When
      you will create more than one pcm instances, specify the
      different numbers in this argument. For example,
      <parameter>index</parameter> = 1 for the second PCM device.  
      </para>
      <para>
        The fourth and fifth arguments are the number of substreams
      for playback and capture, respectively. Here both 1 are given in
      the above example.  When no playback or no capture is available,
      pass 0 to the corresponding argument.
      </para>
      <para>
        If a chip supports multiple playbacks or captures, you can
      specify more numbers, but they must be handled properly in
      open/close, etc. callbacks.  When you need to know which
      substream you are referring to, then it can be obtained from
      struct <structname>snd_pcm_substream</structname> data passed to each callback
      as follows: 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_pcm_substream *substream;
  int index = substream->number;
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        After the pcm is created, you need to set operators for each
        pcm stream. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK,
                  &snd_mychip_playback_ops);
  snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE,
                  &snd_mychip_capture_ops);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The operators are defined typically like this:
        <informalexample>
          <programlisting>
 <![CDATA[
  static struct snd_pcm_ops snd_mychip_playback_ops = {
          .open =        snd_mychip_pcm_open,
          .close =       snd_mychip_pcm_close,
          .ioctl =       snd_pcm_lib_ioctl,
          .hw_params =   snd_mychip_pcm_hw_params,
          .hw_free =     snd_mychip_pcm_hw_free,
          .prepare =     snd_mychip_pcm_prepare,
          .trigger =     snd_mychip_pcm_trigger,
          .pointer =     snd_mychip_pcm_pointer,
  };
 ]]>
          </programlisting>
        </informalexample>
        Each of callbacks is explained in the subsection 
        <link linkend="pcm-interface-operators"><citetitle>
        Operators</citetitle></link>.
      </para>
      <para>
        After setting the operators, most likely you'd like to
        pre-allocate the buffer. For the pre-allocation, simply call
        the following: 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_pcm_lib_preallocate_pages_for_all(pcm, SNDRV_DMA_TYPE_DEV,
                                        snd_dma_pci_data(chip->pci),
                                        64*1024, 64*1024);
 ]]>
          </programlisting>
        </informalexample>
        It will allocate up to 64kB buffer as default. The details of
      buffer management will be described in the later section <link
      linkend="buffer-and-memory"><citetitle>Buffer and Memory
      Management</citetitle></link>. 
      </para>
      <para>
        Additionally, you can set some extra information for this pcm
        in pcm-&gt;info_flags.
        The available values are defined as
        <constant>SNDRV_PCM_INFO_XXX</constant> in
        <filename>&lt;sound/asound.h&gt;</filename>, which is used for
        the hardware definition (described later). When your soundchip
        supports only half-duplex, specify like this: 
        <informalexample>
          <programlisting>
 <![CDATA[
  pcm->info_flags = SNDRV_PCM_INFO_HALF_DUPLEX;
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
    <section id="pcm-interface-destructor">
      <title>... And the Destructor?</title>
      <para>
        The destructor for a pcm instance is not always
      necessary. Since the pcm device will be released by the middle
      layer code automatically, you don't have to call destructor
      explicitly.
      </para>
      <para>
        The destructor would be necessary when you created some
        special records internally and need to release them. In such a
        case, set the destructor function to
        pcm-&gt;private_free: 
        <example>
          <title>PCM Instance with a Destructor</title>
          <programlisting>
 <![CDATA[
  static void mychip_pcm_free(struct snd_pcm *pcm)
  {
          struct mychip *chip = snd_pcm_chip(pcm);
          /* free your own data */
          kfree(chip->my_private_pcm_data);
          // do what you like else
          ....
  }
  static int __devinit snd_mychip_new_pcm(struct mychip *chip)
  {
          struct snd_pcm *pcm;
          ....
          /* allocate your own data */
          chip->my_private_pcm_data = kmalloc(...);
          /* set the destructor */
          pcm->private_data = chip;
          pcm->private_free = mychip_pcm_free;
          ....
  }
 ]]>
          </programlisting>
        </example>
      </para>
    </section>
    <section id="pcm-interface-runtime">
      <title>Runtime Pointer - The Chest of PCM Information</title>
 	<para>
 	  When the PCM substream is opened, a PCM runtime instance is
 	allocated and assigned to the substream. This pointer is
 	accessible via <constant>substream-&gt;runtime</constant>.
 	This runtime pointer holds the various information; it holds
 	the copy of hw_params and sw_params configurations, the buffer
 	pointers, mmap records, spinlocks, etc.  Almost everyhing you
 	need for controlling the PCM can be found there.
 	</para>
 	<para>
 	The definition of runtime instance is found in
 	<filename>&lt;sound/pcm.h&gt;</filename>.  Here is the
 	copy from the file.
          <informalexample>
            <programlisting>
 <![CDATA[
 struct _snd_pcm_runtime {
 	/* -- Status -- */
 	struct snd_pcm_substream *trigger_master;
 	snd_timestamp_t trigger_tstamp;	/* trigger timestamp */
 	int overrange;
 	snd_pcm_uframes_t avail_max;
 	snd_pcm_uframes_t hw_ptr_base;	/* Position at buffer restart */
 	snd_pcm_uframes_t hw_ptr_interrupt; /* Position at interrupt time*/
 	/* -- HW params -- */
 	snd_pcm_access_t access;	/* access mode */
 	snd_pcm_format_t format;	/* SNDRV_PCM_FORMAT_* */
 	snd_pcm_subformat_t subformat;	/* subformat */
 	unsigned int rate;		/* rate in Hz */
 	unsigned int channels;		/* channels */
 	snd_pcm_uframes_t period_size;	/* period size */
 	unsigned int periods;		/* periods */
 	snd_pcm_uframes_t buffer_size;	/* buffer size */
 	unsigned int tick_time;		/* tick time */
 	snd_pcm_uframes_t min_align;	/* Min alignment for the format */
 	size_t byte_align;
 	unsigned int frame_bits;
 	unsigned int sample_bits;
 	unsigned int info;
 	unsigned int rate_num;
 	unsigned int rate_den;
 	/* -- SW params -- */
 	struct timespec tstamp_mode;	/* mmap timestamp is updated */
  	unsigned int period_step;
 	unsigned int sleep_min;		/* min ticks to sleep */
 	snd_pcm_uframes_t xfer_align;	/* xfer size need to be a multiple */
 	snd_pcm_uframes_t start_threshold;
 	snd_pcm_uframes_t stop_threshold;
 	snd_pcm_uframes_t silence_threshold; /* Silence filling happens when
 						noise is nearest than this */
 	snd_pcm_uframes_t silence_size;	/* Silence filling size */
 	snd_pcm_uframes_t boundary;	/* pointers wrap point */
 	snd_pcm_uframes_t silenced_start;
 	snd_pcm_uframes_t silenced_size;
 	snd_pcm_sync_id_t sync;		/* hardware synchronization ID */
 	/* -- mmap -- */
 	volatile struct snd_pcm_mmap_status *status;
 	volatile struct snd_pcm_mmap_control *control;
 	atomic_t mmap_count;
 	/* -- locking / scheduling -- */
 	spinlock_t lock;
 	wait_queue_head_t sleep;
 	struct timer_list tick_timer;
 	struct fasync_struct *fasync;
 	/* -- private section -- */
 	void *private_data;
 	void (*private_free)(struct snd_pcm_runtime *runtime);
 	/* -- hardware description -- */
 	struct snd_pcm_hardware hw;
 	struct snd_pcm_hw_constraints hw_constraints;
 	/* -- interrupt callbacks -- */
 	void (*transfer_ack_begin)(struct snd_pcm_substream *substream);
 	void (*transfer_ack_end)(struct snd_pcm_substream *substream);
 	/* -- timer -- */
 	unsigned int timer_resolution;	/* timer resolution */
 	/* -- DMA -- */           
 	unsigned char *dma_area;	/* DMA area */
 	dma_addr_t dma_addr;		/* physical bus address (not accessible from main CPU) */
 	size_t dma_bytes;		/* size of DMA area */
 	struct snd_dma_buffer *dma_buffer_p;	/* allocated buffer */
 #if defined(CONFIG_SND_PCM_OSS) || defined(CONFIG_SND_PCM_OSS_MODULE)
 	/* -- OSS things -- */
 	struct snd_pcm_oss_runtime oss;
 #endif
 };
 ]]>
            </programlisting>
          </informalexample>
 	</para>
 	<para>
 	  For the operators (callbacks) of each sound driver, most of
 	these records are supposed to be read-only.  Only the PCM
 	middle-layer changes / updates these info.  The exceptions are
 	the hardware description (hw), interrupt callbacks
 	(transfer_ack_xxx), DMA buffer information, and the private
 	data.  Besides, if you use the standard buffer allocation
 	method via <function>snd_pcm_lib_malloc_pages()</function>,
 	you don't need to set the DMA buffer information by yourself.
 	</para>
 	<para>
 	In the sections below, important records are explained.
 	</para>
 	<section id="pcm-interface-runtime-hw">
 	<title>Hardware Description</title>
 	<para>
 	  The hardware descriptor (struct <structname>snd_pcm_hardware</structname>)
 	contains the definitions of the fundamental hardware
 	configuration.  Above all, you'll need to define this in
 	<link linkend="pcm-interface-operators-open-callback"><citetitle>
 	the open callback</citetitle></link>.
 	Note that the runtime instance holds the copy of the
 	descriptor, not the pointer to the existing descriptor.  That
 	is, in the open callback, you can modify the copied descriptor
 	(<constant>runtime-&gt;hw</constant>) as you need.  For example, if the maximum
 	number of channels is 1 only on some chip models, you can
 	still use the same hardware descriptor and change the
 	channels_max later:
          <informalexample>
            <programlisting>
 <![CDATA[
          struct snd_pcm_runtime *runtime = substream->runtime;
          ...
          runtime->hw = snd_mychip_playback_hw; /* common definition */
          if (chip->model == VERY_OLD_ONE)
                  runtime->hw.channels_max = 1;
 ]]>
            </programlisting>
          </informalexample>
 	</para>
 	<para>
 	  Typically, you'll have a hardware descriptor like below:
          <informalexample>
            <programlisting>
 <![CDATA[
  static struct snd_pcm_hardware snd_mychip_playback_hw = {
          .info = (SNDRV_PCM_INFO_MMAP |
                   SNDRV_PCM_INFO_INTERLEAVED |
                   SNDRV_PCM_INFO_BLOCK_TRANSFER |
                   SNDRV_PCM_INFO_MMAP_VALID),
          .formats =          SNDRV_PCM_FMTBIT_S16_LE,
          .rates =            SNDRV_PCM_RATE_8000_48000,
          .rate_min =         8000,
          .rate_max =         48000,
          .channels_min =     2,
          .channels_max =     2,
          .buffer_bytes_max = 32768,
          .period_bytes_min = 4096,
          .period_bytes_max = 32768,
          .periods_min =      1,
          .periods_max =      1024,
  };
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
 	<itemizedlist>
 	<listitem><para>
          The <structfield>info</structfield> field contains the type and
        capabilities of this pcm. The bit flags are defined in
        <filename>&lt;sound/asound.h&gt;</filename> as
        <constant>SNDRV_PCM_INFO_XXX</constant>. Here, at least, you
        have to specify whether the mmap is supported and which
        interleaved format is supported.
        When the mmap is supported, add
        <constant>SNDRV_PCM_INFO_MMAP</constant> flag here. When the
        hardware supports the interleaved or the non-interleaved
        format, <constant>SNDRV_PCM_INFO_INTERLEAVED</constant> or
        <constant>SNDRV_PCM_INFO_NONINTERLEAVED</constant> flag must
        be set, respectively. If both are supported, you can set both,
        too. 
        </para>
        <para>
          In the above example, <constant>MMAP_VALID</constant> and
        <constant>BLOCK_TRANSFER</constant> are specified for OSS mmap
        mode. Usually both are set. Of course,
        <constant>MMAP_VALID</constant> is set only if the mmap is
        really supported. 
        </para>
        <para>
          The other possible flags are
        <constant>SNDRV_PCM_INFO_PAUSE</constant> and
        <constant>SNDRV_PCM_INFO_RESUME</constant>. The
        <constant>PAUSE</constant> bit means that the pcm supports the
        <quote>pause</quote> operation, while the
        <constant>RESUME</constant> bit means that the pcm supports
        the full <quote>suspend/resume</quote> operation.
 	If <constant>PAUSE</constant> flag is set,
 	the <structfield>trigger</structfield> callback below
        must handle the corresponding (pause push/release) commands.
 	The suspend/resume trigger commands can be defined even without
 	<constant>RESUME</constant> flag.  See <link
 	linkend="power-management"><citetitle>
 	Power Management</citetitle></link> section for details.
        </para>
 	<para>
 	  When the PCM substreams can be synchronized (typically,
 	synchorinized start/stop of a playback and a capture streams),
 	you can give <constant>SNDRV_PCM_INFO_SYNC_START</constant>,
 	too.  In this case, you'll need to check the linked-list of
 	PCM substreams in the trigger callback.  This will be
 	described in the later section.
 	</para>
 	</listitem>
 	<listitem>
        <para>
          <structfield>formats</structfield> field contains the bit-flags
        of supported formats (<constant>SNDRV_PCM_FMTBIT_XXX</constant>).
        If the hardware supports more than one format, give all or'ed
        bits.  In the example above, the signed 16bit little-endian
        format is specified.
        </para>
 	</listitem>
 	<listitem>
        <para>
        <structfield>rates</structfield> field contains the bit-flags of
        supported rates (<constant>SNDRV_PCM_RATE_XXX</constant>).
        When the chip supports continuous rates, pass
        <constant>CONTINUOUS</constant> bit additionally.
        The pre-defined rate bits are provided only for typical
 	rates. If your chip supports unconventional rates, you need to add
        <constant>KNOT</constant> bit and set up the hardware
        constraint manually (explained later).
        </para>
 	</listitem>
 	<listitem>
 	<para>
 	<structfield>rate_min</structfield> and
 	<structfield>rate_max</structfield> define the minimal and
 	maximal sample rate.  This should correspond somehow to
 	<structfield>rates</structfield> bits.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	<structfield>channel_min</structfield> and
 	<structfield>channel_max</structfield> 
 	define, as you might already expected, the minimal and maximal
 	number of channels.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	<structfield>buffer_bytes_max</structfield> defines the
 	maximal buffer size in bytes.  There is no
 	<structfield>buffer_bytes_min</structfield> field, since
 	it can be calculated from the minimal period size and the
 	minimal number of periods.
 	Meanwhile, <structfield>period_bytes_min</structfield> and
 	define the minimal and maximal size of the period in bytes.
 	<structfield>periods_max</structfield> and
 	<structfield>periods_min</structfield> define the maximal and
 	minimal number of periods in the buffer.
        </para>
 	<para>
 	The <quote>period</quote> is a term, that corresponds to
 	fragment in the OSS world.  The period defines the size at
 	which the PCM interrupt is generated. This size strongly
 	depends on the hardware. 
 	Generally, the smaller period size will give you more
 	interrupts, that is, more controls. 
 	In the case of capture, this size defines the input latency.
 	On the other hand, the whole buffer size defines the
 	output latency for the playback direction.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	There is also a field <structfield>fifo_size</structfield>.
 	This specifies the size of the hardware FIFO, but it's not
 	used currently in the driver nor in the alsa-lib.  So, you
 	can ignore this field.
 	</para>
 	</listitem>
 	</itemizedlist>
 	</para>
 	</section>
 	<section id="pcm-interface-runtime-config">
 	<title>PCM Configurations</title>
 	<para>
 	Ok, let's go back again to the PCM runtime records.
 	The most frequently referred records in the runtime instance are
 	the PCM configurations.
 	The PCM configurations are stored on runtime instance
 	after the application sends <type>hw_params</type> data via
 	alsa-lib.  There are many fields copied from hw_params and
 	sw_params structs.  For example,
 	<structfield>format</structfield> holds the format type
 	chosen by the application.  This field contains the enum value
 	<constant>SNDRV_PCM_FORMAT_XXX</constant>.
 	</para>
 	<para>
 	One thing to be noted is that the configured buffer and period
 	sizes are stored in <quote>frames</quote> in the runtime
        In the ALSA world, 1 frame = channels * samples-size.
 	For conversion between frames and bytes, you can use the
 	helper functions, <function>frames_to_bytes()</function> and
          <function>bytes_to_frames()</function>. 
          <informalexample>
            <programlisting>
 <![CDATA[
  period_bytes = frames_to_bytes(runtime, runtime->period_size);
 ]]>
            </programlisting>
          </informalexample>
        </para>
 	<para>
 	Also, many software parameters (sw_params) are
 	stored in frames, too.  Please check the type of the field.
 	<type>snd_pcm_uframes_t</type> is for the frames as unsigned
 	integer while <type>snd_pcm_sframes_t</type> is for the frames
 	as signed integer.
 	</para>
 	</section>
 	<section id="pcm-interface-runtime-dma">
 	<title>DMA Buffer Information</title>
 	<para>
 	The DMA buffer is defined by the following four fields,
 	<structfield>dma_area</structfield>,
 	<structfield>dma_addr</structfield>,
 	<structfield>dma_bytes</structfield> and
 	<structfield>dma_private</structfield>.
 	The <structfield>dma_area</structfield> holds the buffer
 	pointer (the logical address).  You can call
 	<function>memcpy</function> from/to 
 	this pointer.  Meanwhile, <structfield>dma_addr</structfield>
 	holds the physical address of the buffer.  This field is
 	specified only when the buffer is a linear buffer.
 	<structfield>dma_bytes</structfield> holds the size of buffer
 	in bytes.  <structfield>dma_private</structfield> is used for
 	the ALSA DMA allocator.
 	</para>
 	<para>
 	If you use a standard ALSA function,
 	<function>snd_pcm_lib_malloc_pages()</function>, for
 	allocating the buffer, these fields are set by the ALSA middle
 	layer, and you should <emphasis>not</emphasis> change them by
 	yourself.  You can read them but not write them.
 	On the other hand, if you want to allocate the buffer by
 	yourself, you'll need to manage it in hw_params callback.
 	At least, <structfield>dma_bytes</structfield> is mandatory.
 	<structfield>dma_area</structfield> is necessary when the
 	buffer is mmapped.  If your driver doesn't support mmap, this
 	field is not necessary.  <structfield>dma_addr</structfield>
 	is also not mandatory.  You can use
 	<structfield>dma_private</structfield> as you like, too.
 	</para>
 	</section>
 	<section id="pcm-interface-runtime-status">
 	<title>Running Status</title>
 	<para>
 	The running status can be referred via <constant>runtime-&gt;status</constant>.
 	This is the pointer to struct <structname>snd_pcm_mmap_status</structname>
 	record.  For example, you can get the current DMA hardware
 	pointer via <constant>runtime-&gt;status-&gt;hw_ptr</constant>.
 	</para>
 	<para>
 	The DMA application pointer can be referred via
 	<constant>runtime-&gt;control</constant>, which points
 	struct <structname>snd_pcm_mmap_control</structname> record.
 	However, accessing directly to this value is not recommended.
 	</para>
 	</section>
 	<section id="pcm-interface-runtime-private">
 	<title>Private Data</title> 
 	<para>
 	You can allocate a record for the substream and store it in
 	<constant>runtime-&gt;private_data</constant>.  Usually, this
 	done in
 	<link linkend="pcm-interface-operators-open-callback"><citetitle>
 	the open callback</citetitle></link>.
 	Don't mix this with <constant>pcm-&gt;private_data</constant>.
 	The <constant>pcm-&gt;private_data</constant> usually points the
 	chip instance assigned statically at the creation of PCM, while the 
 	<constant>runtime-&gt;private_data</constant> points a dynamic
 	data created at the PCM open callback.
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_open(struct snd_pcm_substream *substream)
  {
          struct my_pcm_data *data;
          ....
          data = kmalloc(sizeof(*data), GFP_KERNEL);
          substream->runtime->private_data = data;
          ....
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          The allocated object must be released in
 	<link linkend="pcm-interface-operators-open-callback"><citetitle>
 	the close callback</citetitle></link>.
        </para>
 	</section>
 	<section id="pcm-interface-runtime-intr">
 	<title>Interrupt Callbacks</title>
 	<para>
 	The field <structfield>transfer_ack_begin</structfield> and
 	<structfield>transfer_ack_end</structfield> are called at
 	the beginning and the end of
 	<function>snd_pcm_period_elapsed()</function>, respectively. 
 	</para>
 	</section>
    </section>
    <section id="pcm-interface-operators">
      <title>Operators</title>
      <para>
        OK, now let me explain the detail of each pcm callback
      (<parameter>ops</parameter>). In general, every callback must
      return 0 if successful, or a negative number with the error
      number such as <constant>-EINVAL</constant> at any
      error. 
      </para>
      <para>
        The callback function takes at least the argument with
        <structname>snd_pcm_substream</structname> pointer. For retrieving the
        chip record from the given substream instance, you can use the
        following macro. 
        <informalexample>
          <programlisting>
 <![CDATA[
  int xxx() {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          ....
  }
 ]]>
          </programlisting>
        </informalexample>
 	The macro reads <constant>substream-&gt;private_data</constant>,
 	which is a copy of <constant>pcm-&gt;private_data</constant>.
 	You can override the former if you need to assign different data
 	records per PCM substream.  For example, cmi8330 driver assigns
 	different private_data for playback and capture directions,
 	because it uses two different codecs (SB- and AD-compatible) for
 	different directions.
      </para>
      <section id="pcm-interface-operators-open-callback">
        <title>open callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_open(struct snd_pcm_substream *substream);
 ]]>
            </programlisting>
          </informalexample>
          This is called when a pcm substream is opened.
        </para>
        <para>
          At least, here you have to initialize the runtime-&gt;hw
          record. Typically, this is done by like this: 
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_open(struct snd_pcm_substream *substream)
  {
          struct mychip *chip = snd_pcm_substream_chip(substream);
          struct snd_pcm_runtime *runtime = substream->runtime;
          runtime->hw = snd_mychip_playback_hw;
          return 0;
  }
 ]]>
            </programlisting>
          </informalexample>
          where <parameter>snd_mychip_playback_hw</parameter> is the
          pre-defined hardware description.
 	</para>
 	<para>
 	You can allocate a private data in this callback, as described
 	in <link linkend="pcm-interface-runtime-private"><citetitle>
 	Private Data</citetitle></link> section.
 	</para>
 	<para>
 	If the hardware configuration needs more constraints, set the
 	hardware constraints here, too.
 	See <link linkend="pcm-interface-constraints"><citetitle>
 	Constraints</citetitle></link> for more details.
 	</para>
      </section>
      <section id="pcm-interface-operators-close-callback">
        <title>close callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_close(struct snd_pcm_substream *substream);
 ]]>
            </programlisting>
          </informalexample>
          Obviously, this is called when a pcm substream is closed.
        </para>
        <para>
          Any private instance for a pcm substream allocated in the
          open callback will be released here. 
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_close(struct snd_pcm_substream *substream)
  {
          ....
          kfree(substream->runtime->private_data);
          ....
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
      </section>
      <section id="pcm-interface-operators-ioctl-callback">
        <title>ioctl callback</title>
        <para>
          This is used for any special action to pcm ioctls. But
        usually you can pass a generic ioctl callback, 
        <function>snd_pcm_lib_ioctl</function>.
        </para>
      </section>
      <section id="pcm-interface-operators-hw-params-callback">
        <title>hw_params callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_hw_params(struct snd_pcm_substream *substream,
                               struct snd_pcm_hw_params *hw_params);
 ]]>
            </programlisting>
          </informalexample>
          This and <structfield>hw_free</structfield> callbacks exist
        only on ALSA 0.9.x. 
        </para>
        <para>
          This is called when the hardware parameter
        (<structfield>hw_params</structfield>) is set
        up by the application, 
        that is, once when the buffer size, the period size, the
        format, etc. are defined for the pcm substream. 
        </para>
        <para>
          Many hardware set-up should be done in this callback,
        including the allocation of buffers. 
        </para>
        <para>
          Parameters to be initialized are retrieved by
          <function>params_xxx()</function> macros. For allocating a
          buffer, you can call a helper function, 
          <informalexample>
            <programlisting>
 <![CDATA[
  snd_pcm_lib_malloc_pages(substream, params_buffer_bytes(hw_params));
 ]]>
            </programlisting>
          </informalexample>
          <function>snd_pcm_lib_malloc_pages()</function> is available
 	  only when the DMA buffers have been pre-allocated.
 	  See the section <link
 	  linkend="buffer-and-memory-buffer-types"><citetitle>
 	  Buffer Types</citetitle></link> for more details.
        </para>
        <para>
          Note that this and <structfield>prepare</structfield> callbacks
        may be called multiple times per initialization.
        For example, the OSS emulation may
        call these callbacks at each change via its ioctl. 
        </para>
        <para>
          Thus, you need to take care not to allocate the same buffers
        many times, which will lead to memory leak!  Calling the
        helper function above many times is OK. It will release the
        previous buffer automatically when it was already allocated. 
        </para>
        <para>
          Another note is that this callback is non-atomic
        (schedulable). This is important, because the
        <structfield>trigger</structfield> callback 
        is atomic (non-schedulable). That is, mutex or any
        schedule-related functions are not available in
        <structfield>trigger</structfield> callback.
 	Please see the subsection
 	<link linkend="pcm-interface-atomicity"><citetitle>
 	Atomicity</citetitle></link> for details.
        </para>
      </section>
      <section id="pcm-interface-operators-hw-free-callback">
        <title>hw_free callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_hw_free(struct snd_pcm_substream *substream);
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          This is called to release the resources allocated via
          <structfield>hw_params</structfield>. For example, releasing the
          buffer via 
          <function>snd_pcm_lib_malloc_pages()</function> is done by
          calling the following: 
          <informalexample>
            <programlisting>
 <![CDATA[
  snd_pcm_lib_free_pages(substream);
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          This function is always called before the close callback is called.
          Also, the callback may be called multiple times, too.
          Keep track whether the resource was already released. 
        </para>
      </section>
      <section id="pcm-interface-operators-prepare-callback">
       <title>prepare callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_prepare(struct snd_pcm_substream *substream);
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          This callback is called when the pcm is
        <quote>prepared</quote>. You can set the format type, sample
        rate, etc. here. The difference from
        <structfield>hw_params</structfield> is that the 
        <structfield>prepare</structfield> callback will be called at each
        time 
        <function>snd_pcm_prepare()</function> is called, i.e. when
        recovered after underruns, etc. 
        </para>
        <para>
 	Note that this callback became non-atomic since the recent version.
 	You can use schedule-related functions safely in this callback now.
        </para>
        <para>
          In this and the following callbacks, you can refer to the
        values via the runtime record,
        substream-&gt;runtime.
        For example, to get the current
        rate, format or channels, access to
        runtime-&gt;rate,
        runtime-&gt;format or
        runtime-&gt;channels, respectively. 
        The physical address of the allocated buffer is set to
 	runtime-&gt;dma_area.  The buffer and period sizes are
 	in runtime-&gt;buffer_size and runtime-&gt;period_size,
 	respectively.
        </para>
        <para>
          Be careful that this callback will be called many times at
        each set up, too. 
        </para>
      </section>
      <section id="pcm-interface-operators-trigger-callback">
        <title>trigger callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_xxx_trigger(struct snd_pcm_substream *substream, int cmd);
 ]]>
            </programlisting>
          </informalexample>
          This is called when the pcm is started, stopped or paused.
        </para>
        <para>
          Which action is specified in the second argument,
          <constant>SNDRV_PCM_TRIGGER_XXX</constant> in
          <filename>&lt;sound/pcm.h&gt;</filename>. At least,
          <constant>START</constant> and <constant>STOP</constant>
          commands must be defined in this callback. 
          <informalexample>
            <programlisting>
 <![CDATA[
  switch (cmd) {
  case SNDRV_PCM_TRIGGER_START:
          // do something to start the PCM engine
          break;
  case SNDRV_PCM_TRIGGER_STOP:
          // do something to stop the PCM engine
          break;
  default:
          return -EINVAL;
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
          When the pcm supports the pause operation (given in info
        field of the hardware table), <constant>PAUSE_PUSE</constant>
        and <constant>PAUSE_RELEASE</constant> commands must be
        handled here, too. The former is the command to pause the pcm,
        and the latter to restart the pcm again. 
        </para>
        <para>
          When the pcm supports the suspend/resume operation,
 	regardless of full or partial suspend/resume support,
        <constant>SUSPEND</constant> and <constant>RESUME</constant>
        commands must be handled, too.
        These commands are issued when the power-management status is
        changed.  Obviously, the <constant>SUSPEND</constant> and
        <constant>RESUME</constant>
        do suspend and resume of the pcm substream, and usually, they
        are identical with <constant>STOP</constant> and
        <constant>START</constant> commands, respectively.
 	  See <link linkend="power-management"><citetitle>
 	Power Management</citetitle></link> section for details.
        </para>
        <para>
          As mentioned, this callback is atomic.  You cannot call
 	  the function going to sleep.
 	  The trigger callback should be as minimal as possible,
 	  just really triggering the DMA.  The other stuff should be
 	  initialized hw_params and prepare callbacks properly
 	  beforehand.
        </para>
      </section>
      <section id="pcm-interface-operators-pointer-callback">
        <title>pointer callback</title>
        <para>
          <informalexample>
            <programlisting>
 <![CDATA[
  static snd_pcm_uframes_t snd_xxx_pointer(struct snd_pcm_substream *substream)
 ]]>
            </programlisting>
          </informalexample>
          This callback is called when the PCM middle layer inquires
        the current hardware position on the buffer. The position must
        be returned in frames (which was in bytes on ALSA 0.5.x),
        ranged from 0 to buffer_size - 1.
        </para>
        <para>
          This is called usually from the buffer-update routine in the
        pcm middle layer, which is invoked when
        <function>snd_pcm_period_elapsed()</function> is called in the
        interrupt routine. Then the pcm middle layer updates the
        position and calculates the available space, and wakes up the
        sleeping poll threads, etc. 
        </para>
        <para>
          This callback is also atomic.
        </para>
      </section>
      <section id="pcm-interface-operators-copy-silence">
        <title>copy and silence callbacks</title>
        <para>
          These callbacks are not mandatory, and can be omitted in
        most cases. These callbacks are used when the hardware buffer
        cannot be on the normal memory space. Some chips have their
        own buffer on the hardware which is not mappable. In such a
        case, you have to transfer the data manually from the memory
        buffer to the hardware buffer. Or, if the buffer is
        non-contiguous on both physical and virtual memory spaces,
        these callbacks must be defined, too. 
        </para>
        <para>
          If these two callbacks are defined, copy and set-silence
        operations are done by them. The detailed will be described in
        the later section <link
        linkend="buffer-and-memory"><citetitle>Buffer and Memory
        Management</citetitle></link>. 
        </para>
      </section>
      <section id="pcm-interface-operators-ack">
        <title>ack callback</title>
        <para>
          This callback is also not mandatory. This callback is called
        when the appl_ptr is updated in read or write operations.
        Some drivers like emu10k1-fx and cs46xx need to track the
 	current appl_ptr for the internal buffer, and this callback
 	is useful only for such a purpose.
 	</para>
 	<para>
 	  This callback is atomic.
 	</para>
      </section>
      <section id="pcm-interface-operators-page-callback">
        <title>page callback</title>
        <para>
          This callback is also not mandatory. This callback is used
        mainly for the non-contiguous buffer. The mmap calls this
        callback to get the page address. Some examples will be
        explained in the later section <link
        linkend="buffer-and-memory"><citetitle>Buffer and Memory
        Management</citetitle></link>, too. 
        </para>
      </section>
    </section>
    <section id="pcm-interface-interrupt-handler">
      <title>Interrupt Handler</title>
      <para>
        The rest of pcm stuff is the PCM interrupt handler. The
      role of PCM interrupt handler in the sound driver is to update
      the buffer position and to tell the PCM middle layer when the
      buffer position goes across the prescribed period size. To
      inform this, call <function>snd_pcm_period_elapsed()</function>
      function. 
      </para>
      <para>
        There are several types of sound chips to generate the interrupts.
      </para>
      <section id="pcm-interface-interrupt-handler-boundary">
        <title>Interrupts at the period (fragment) boundary</title>
        <para>
          This is the most frequently found type:  the hardware
        generates an interrupt at each period boundary.
 	In this case, you can call
        <function>snd_pcm_period_elapsed()</function> at each 
        interrupt. 
        </para>
        <para>
          <function>snd_pcm_period_elapsed()</function> takes the
        substream pointer as its argument. Thus, you need to keep the
        substream pointer accessible from the chip instance. For
        example, define substream field in the chip record to hold the
        current running substream pointer, and set the pointer value
        at open callback (and reset at close callback). 
        </para>
        <para>
          If you acquire a spinlock in the interrupt handler, and the
        lock is used in other pcm callbacks, too, then you have to
        release the lock before calling
        <function>snd_pcm_period_elapsed()</function>, because
        <function>snd_pcm_period_elapsed()</function> calls other pcm
        callbacks inside. 
        </para>
        <para>
          A typical coding would be like:
          <example>
 	    <title>Interrupt Handler Case #1</title>
            <programlisting>
 <![CDATA[
  static irqreturn_t snd_mychip_interrupt(int irq, void *dev_id,
                                          struct pt_regs *regs)
  {
          struct mychip *chip = dev_id;
          spin_lock(&chip->lock);
          ....
          if (pcm_irq_invoked(chip)) {
                  /* call updater, unlock before it */
                  spin_unlock(&chip->lock);
                  snd_pcm_period_elapsed(chip->substream);
                  spin_lock(&chip->lock);
                  // acknowledge the interrupt if necessary
          }
          ....
          spin_unlock(&chip->lock);
          return IRQ_HANDLED;
  }
 ]]>
            </programlisting>
          </example>
        </para>
      </section>
      <section id="pcm-interface-interrupt-handler-timer">
        <title>High-frequent timer interrupts</title>
        <para>
 	This is the case when the hardware doesn't generate interrupts
        at the period boundary but do timer-interrupts at the fixed
        timer rate (e.g. es1968 or ymfpci drivers). 
        In this case, you need to check the current hardware
        position and accumulates the processed sample length at each
        interrupt.  When the accumulated size overcomes the period
        size, call 
        <function>snd_pcm_period_elapsed()</function> and reset the
        accumulator. 
        </para>
        <para>
          A typical coding would be like the following.
          <example>
 	    <title>Interrupt Handler Case #2</title>
            <programlisting>
 <![CDATA[
  static irqreturn_t snd_mychip_interrupt(int irq, void *dev_id,
                                          struct pt_regs *regs)
  {
          struct mychip *chip = dev_id;
          spin_lock(&chip->lock);
          ....
          if (pcm_irq_invoked(chip)) {
                  unsigned int last_ptr, size;
                  /* get the current hardware pointer (in frames) */
                  last_ptr = get_hw_ptr(chip);
                  /* calculate the processed frames since the
                   * last update
                   */
                  if (last_ptr < chip->last_ptr)
                          size = runtime->buffer_size + last_ptr 
                                   - chip->last_ptr; 
                  else
                          size = last_ptr - chip->last_ptr;
                  /* remember the last updated point */
                  chip->last_ptr = last_ptr;
                  /* accumulate the size */
                  chip->size += size;
                  /* over the period boundary? */
                  if (chip->size >= runtime->period_size) {
                          /* reset the accumulator */
                          chip->size %= runtime->period_size;
                          /* call updater */
                          spin_unlock(&chip->lock);
                          snd_pcm_period_elapsed(substream);
                          spin_lock(&chip->lock);
                  }
                  // acknowledge the interrupt if necessary
          }
          ....
          spin_unlock(&chip->lock);
          return IRQ_HANDLED;
  }
 ]]>
            </programlisting>
          </example>
        </para>
      </section>
      <section id="pcm-interface-interrupt-handler-both">
        <title>On calling <function>snd_pcm_period_elapsed()</function></title>
        <para>
          In both cases, even if more than one period are elapsed, you
        don't have to call
        <function>snd_pcm_period_elapsed()</function> many times. Call
        only once. And the pcm layer will check the current hardware
        pointer and update to the latest status. 
        </para>
      </section>
    </section>
    <section id="pcm-interface-atomicity">
      <title>Atomicity</title>
      <para>
      One of the most important (and thus difficult to debug) problem
      on the kernel programming is the race condition.
      On linux kernel, usually it's solved via spin-locks or
      semaphores.  In general, if the race condition may
      happen in the interrupt handler, it's handled as atomic, and you
      have to use spinlock for protecting the critical session.  If it
      never happens in the interrupt and it may take relatively long
      time, you should use semaphore.
      </para>
      <para>
      As already seen, some pcm callbacks are atomic and some are
      not.  For example, <parameter>hw_params</parameter> callback is
      non-atomic, while <parameter>trigger</parameter> callback is
      atomic.  This means, the latter is called already in a spinlock
      held by the PCM middle layer. Please take this atomicity into
      account when you use a spinlock or a semaphore in the callbacks.
      </para>
      <para>
      In the atomic callbacks, you cannot use functions which may call
      <function>schedule</function> or go to
      <function>sleep</function>.  The semaphore and mutex do sleep,
      and hence they cannot be used inside the atomic callbacks
      (e.g. <parameter>trigger</parameter> callback).
      For taking a certain delay in such a callback, please use
      <function>udelay()</function> or <function>mdelay()</function>.
      </para>
      <para>
      All three atomic callbacks (trigger, pointer, and ack) are
      called with local interrupts disabled.
      </para>
    </section>
    <section id="pcm-interface-constraints">
      <title>Constraints</title>
      <para>
        If your chip supports unconventional sample rates, or only the
      limited samples, you need to set a constraint for the
      condition. 
      </para>
      <para>
        For example, in order to restrict the sample rates in the some
        supported values, use
 	<function>snd_pcm_hw_constraint_list()</function>.
 	You need to call this function in the open callback.
        <example>
 	  <title>Example of Hardware Constraints</title>
          <programlisting>
 <![CDATA[
  static unsigned int rates[] =
          {4000, 10000, 22050, 44100};
  static struct snd_pcm_hw_constraint_list constraints_rates = {
          .count = ARRAY_SIZE(rates),
          .list = rates,
          .mask = 0,
  };
  static int snd_mychip_pcm_open(struct snd_pcm_substream *substream)
  {
          int err;
          ....
          err = snd_pcm_hw_constraint_list(substream->runtime, 0,
                                           SNDRV_PCM_HW_PARAM_RATE,
                                           &constraints_rates);
          if (err < 0)
                  return err;
          ....
  }
 ]]>
          </programlisting>
        </example>
      </para>
      <para>
        There are many different constraints.
        Look in <filename>sound/pcm.h</filename> for a complete list.
        You can even define your own constraint rules.
        For example, let's suppose my_chip can manage a substream of 1 channel
        if and only if the format is S16_LE, otherwise it supports any format
        specified in the <structname>snd_pcm_hardware</structname> stucture (or in any
        other constraint_list). You can build a rule like this:
        <example>
 	  <title>Example of Hardware Constraints for Channels</title>
 	  <programlisting>
 <![CDATA[
  static int hw_rule_format_by_channels(struct snd_pcm_hw_params *params,
                                        struct snd_pcm_hw_rule *rule)
  {
          struct snd_interval *c = hw_param_interval(params,
                SNDRV_PCM_HW_PARAM_CHANNELS);
          struct snd_mask *f = hw_param_mask(params, SNDRV_PCM_HW_PARAM_FORMAT);
          struct snd_mask fmt;
          snd_mask_any(&fmt);    /* Init the struct */
          if (c->min < 2) {
                  fmt.bits[0] &= SNDRV_PCM_FMTBIT_S16_LE;
                  return snd_mask_refine(f, &fmt);
          }
          return 0;
  }
 ]]>
          </programlisting>
        </example>
      </para>
      <para>
        Then you need to call this function to add your rule:
       <informalexample>
 	 <programlisting>
 <![CDATA[
  snd_pcm_hw_rule_add(substream->runtime, 0, SNDRV_PCM_HW_PARAM_CHANNELS,
                      hw_rule_channels_by_format, 0, SNDRV_PCM_HW_PARAM_FORMAT,
                      -1);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The rule function is called when an application sets the number of
        channels. But an application can set the format before the number of
        channels. Thus you also need to define the inverse rule:
       <example>
 	 <title>Example of Hardware Constraints for Channels</title>
 	 <programlisting>
 <![CDATA[
  static int hw_rule_channels_by_format(struct snd_pcm_hw_params *params,
                                        struct snd_pcm_hw_rule *rule)
  {
          struct snd_interval *c = hw_param_interval(params,
                        SNDRV_PCM_HW_PARAM_CHANNELS);
          struct snd_mask *f = hw_param_mask(params, SNDRV_PCM_HW_PARAM_FORMAT);
          struct snd_interval ch;
          snd_interval_any(&ch);
          if (f->bits[0] == SNDRV_PCM_FMTBIT_S16_LE) {
                  ch.min = ch.max = 1;
                  ch.integer = 1;
                  return snd_interval_refine(c, &ch);
          }
          return 0;
  }
 ]]>
          </programlisting>
        </example>
      </para>
      <para>
      ...and in the open callback:
       <informalexample>
 	 <programlisting>
 <![CDATA[
  snd_pcm_hw_rule_add(substream->runtime, 0, SNDRV_PCM_HW_PARAM_FORMAT,
                      hw_rule_format_by_channels, 0, SNDRV_PCM_HW_PARAM_CHANNELS,
                      -1);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        I won't explain more details here, rather I
        would like to say, <quote>Luke, use the source.</quote>
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Control Interface  -->
 <!-- ****************************************************** -->
  <chapter id="control-interface">
    <title>Control Interface</title>
    <section id="control-interface-general">
      <title>General</title>
      <para>
        The control interface is used widely for many switches,
      sliders, etc. which are accessed from the user-space. Its most
      important use is the mixer interface. In other words, on ALSA
      0.9.x, all the mixer stuff is implemented on the control kernel
      API (while there was an independent mixer kernel API on 0.5.x). 
      </para>
      <para>
        ALSA has a well-defined AC97 control module. If your chip
      supports only the AC97 and nothing else, you can skip this
      section. 
      </para>
      <para>
        The control API is defined in
      <filename>&lt;sound/control.h&gt;</filename>.
      Include this file if you add your own controls.
      </para>
    </section>
    <section id="control-interface-definition">
      <title>Definition of Controls</title>
      <para>
        For creating a new control, you need to define the three
      callbacks: <structfield>info</structfield>,
      <structfield>get</structfield> and
      <structfield>put</structfield>. Then, define a
      struct <structname>snd_kcontrol_new</structname> record, such as: 
        <example>
 	  <title>Definition of a Control</title>
          <programlisting>
 <![CDATA[
  static struct snd_kcontrol_new my_control __devinitdata = {
          .iface = SNDRV_CTL_ELEM_IFACE_MIXER,
          .name = "PCM Playback Switch",
          .index = 0,
          .access = SNDRV_CTL_ELEM_ACCESS_READWRITE,
          .private_value = 0xffff,
          .info = my_control_info,
          .get = my_control_get,
          .put = my_control_put
  };
 ]]>
          </programlisting>
        </example>
      </para>
      <para>
        Most likely the control is created via
      <function>snd_ctl_new1()</function>, and in such a case, you can
      add <parameter>__devinitdata</parameter> prefix to the
      definition like above. 
      </para>
      <para>
        The <structfield>iface</structfield> field specifies the type of
      the control, <constant>SNDRV_CTL_ELEM_IFACE_XXX</constant>, which
      is usually <constant>MIXER</constant>.
      Use <constant>CARD</constant> for global controls that are not
      logically part of the mixer.
      If the control is closely associated with some specific device on
      the sound card, use <constant>HWDEP</constant>,
      <constant>PCM</constant>, <constant>RAWMIDI</constant>,
      <constant>TIMER</constant>, or <constant>SEQUENCER</constant>, and
      specify the device number with the
      <structfield>device</structfield> and
      <structfield>subdevice</structfield> fields.
      </para>
      <para>
        The <structfield>name</structfield> is the name identifier
      string. On ALSA 0.9.x, the control name is very important,
      because its role is classified from its name. There are
      pre-defined standard control names. The details are described in
      the subsection
      <link linkend="control-interface-control-names"><citetitle>
      Control Names</citetitle></link>.
      </para>
      <para>
        The <structfield>index</structfield> field holds the index number
      of this control. If there are several different controls with
      the same name, they can be distinguished by the index
      number. This is the case when 
      several codecs exist on the card. If the index is zero, you can
      omit the definition above. 
      </para>
      <para>
        The <structfield>access</structfield> field contains the access
      type of this control. Give the combination of bit masks,
      <constant>SNDRV_CTL_ELEM_ACCESS_XXX</constant>, there.
      The detailed will be explained in the subsection
      <link linkend="control-interface-access-flags"><citetitle>
      Access Flags</citetitle></link>.
      </para>
      <para>
        The <structfield>private_value</structfield> field contains
      an arbitrary long integer value for this record. When using
      generic <structfield>info</structfield>,
      <structfield>get</structfield> and
      <structfield>put</structfield> callbacks, you can pass a value 
      through this field. If several small numbers are necessary, you can
      combine them in bitwise. Or, it's possible to give a pointer
      (casted to unsigned long) of some record to this field, too. 
      </para>
      <para>
        The other three are
 	<link linkend="control-interface-callbacks"><citetitle>
 	callback functions</citetitle></link>.
      </para>
    </section>
    <section id="control-interface-control-names">
      <title>Control Names</title>
      <para>
        There are some standards for defining the control names. A
      control is usually defined from the three parts as
      <quote>SOURCE DIRECTION FUNCTION</quote>. 
      </para>
      <para>
        The first, <constant>SOURCE</constant>, specifies the source
      of the control, and is a string such as <quote>Master</quote>,
      <quote>PCM</quote>, <quote>CD</quote> or
      <quote>Line</quote>. There are many pre-defined sources. 
      </para>
      <para>
        The second, <constant>DIRECTION</constant>, is one of the
      following strings according to the direction of the control:
      <quote>Playback</quote>, <quote>Capture</quote>, <quote>Bypass
      Playback</quote> and <quote>Bypass Capture</quote>. Or, it can
      be omitted, meaning both playback and capture directions. 
      </para>
      <para>
        The third, <constant>FUNCTION</constant>, is one of the
      following strings according to the function of the control:
      <quote>Switch</quote>, <quote>Volume</quote> and
      <quote>Route</quote>. 
      </para>
      <para>
        The example of control names are, thus, <quote>Master Capture
      Switch</quote> or <quote>PCM Playback Volume</quote>. 
      </para>
      <para>
        There are some exceptions:
      </para>
      <section id="control-interface-control-names-global">
        <title>Global capture and playback</title>
        <para>
          <quote>Capture Source</quote>, <quote>Capture Switch</quote>
        and <quote>Capture Volume</quote> are used for the global
        capture (input) source, switch and volume. Similarly,
        <quote>Playback Switch</quote> and <quote>Playback
        Volume</quote> are used for the global output gain switch and
        volume. 
        </para>
      </section>
      <section id="control-interface-control-names-tone">
        <title>Tone-controls</title>
        <para>
          tone-control switch and volumes are specified like
        <quote>Tone Control - XXX</quote>, e.g. <quote>Tone Control -
        Switch</quote>, <quote>Tone Control - Bass</quote>,
        <quote>Tone Control - Center</quote>.  
        </para>
      </section>
      <section id="control-interface-control-names-3d">
        <title>3D controls</title>
        <para>
          3D-control switches and volumes are specified like <quote>3D
        Control - XXX</quote>, e.g. <quote>3D Control -
        Switch</quote>, <quote>3D Control - Center</quote>, <quote>3D
        Control - Space</quote>. 
        </para>
      </section>
      <section id="control-interface-control-names-mic">
        <title>Mic boost</title>
        <para>
          Mic-boost switch is set as <quote>Mic Boost</quote> or
        <quote>Mic Boost (6dB)</quote>. 
        </para>
        <para>
          More precise information can be found in
        <filename>Documentation/sound/alsa/ControlNames.txt</filename>.
        </para>
      </section>
    </section>
    <section id="control-interface-access-flags">
      <title>Access Flags</title>
      <para>
      The access flag is the bit-flags which specifies the access type
      of the given control.  The default access type is
      <constant>SNDRV_CTL_ELEM_ACCESS_READWRITE</constant>, 
      which means both read and write are allowed to this control.
      When the access flag is omitted (i.e. = 0), it is
      regarded as <constant>READWRITE</constant> access as default. 
      </para>
      <para>
      When the control is read-only, pass
      <constant>SNDRV_CTL_ELEM_ACCESS_READ</constant> instead.
      In this case, you don't have to define
      <structfield>put</structfield> callback.
      Similarly, when the control is write-only (although it's a rare
      case), you can use <constant>WRITE</constant> flag instead, and
      you don't need <structfield>get</structfield> callback.
      </para>
      <para>
      If the control value changes frequently (e.g. the VU meter),
      <constant>VOLATILE</constant> flag should be given.  This means
      that the control may be changed without
      <link linkend="control-interface-change-notification"><citetitle>
      notification</citetitle></link>.  Applications should poll such
      a control constantly.
      </para>
      <para>
      When the control is inactive, set
      <constant>INACTIVE</constant> flag, too.
      There are <constant>LOCK</constant> and
      <constant>OWNER</constant> flags for changing the write
      permissions.
      </para>
    </section>
    <section id="control-interface-callbacks">
      <title>Callbacks</title>
      <section id="control-interface-callbacks-info">
        <title>info callback</title>
        <para>
          The <structfield>info</structfield> callback is used to get
        the detailed information of this control. This must store the
        values of the given struct <structname>snd_ctl_elem_info</structname>
        object. For example, for a boolean control with a single
        element will be: 
          <example>
 	    <title>Example of info callback</title>
            <programlisting>
 <![CDATA[
  static int snd_myctl_info(struct snd_kcontrol *kcontrol,
                          struct snd_ctl_elem_info *uinfo)
  {
          uinfo->type = SNDRV_CTL_ELEM_TYPE_BOOLEAN;
          uinfo->count = 1;
          uinfo->value.integer.min = 0;
          uinfo->value.integer.max = 1;
          return 0;
  }
 ]]>
            </programlisting>
          </example>
        </para>
        <para>
          The <structfield>type</structfield> field specifies the type
        of the control. There are <constant>BOOLEAN</constant>,
        <constant>INTEGER</constant>, <constant>ENUMERATED</constant>,
        <constant>BYTES</constant>, <constant>IEC958</constant> and
        <constant>INTEGER64</constant>. The
        <structfield>count</structfield> field specifies the 
        number of elements in this control. For example, a stereo
        volume would have count = 2. The
        <structfield>value</structfield> field is a union, and 
        the values stored are depending on the type. The boolean and
        integer are identical. 
        </para>
        <para>
          The enumerated type is a bit different from others.  You'll
          need to set the string for the currently given item index. 
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_myctl_info(struct snd_kcontrol *kcontrol,
                          struct snd_ctl_elem_info *uinfo)
  {
          static char *texts[4] = {
                  "First", "Second", "Third", "Fourth"
          };
          uinfo->type = SNDRV_CTL_ELEM_TYPE_ENUMERATED;
          uinfo->count = 1;
          uinfo->value.enumerated.items = 4;
          if (uinfo->value.enumerated.item > 3)
                  uinfo->value.enumerated.item = 3;
          strcpy(uinfo->value.enumerated.name,
                 texts[uinfo->value.enumerated.item]);
          return 0;
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
      </section>
      <section id="control-interface-callbacks-get">
        <title>get callback</title>
        <para>
          This callback is used to read the current value of the
        control and to return to the user-space. 
        </para>
        <para>
          For example,
          <example>
 	    <title>Example of get callback</title>
            <programlisting>
 <![CDATA[
  static int snd_myctl_get(struct snd_kcontrol *kcontrol,
                           struct snd_ctl_elem_value *ucontrol)
  {
          struct mychip *chip = snd_kcontrol_chip(kcontrol);
          ucontrol->value.integer.value[0] = get_some_value(chip);
          return 0;
  }
 ]]>
            </programlisting>
          </example>
        </para>
        <para>
          Here, the chip instance is retrieved via
        <function>snd_kcontrol_chip()</function> macro.  This macro
        just accesses to kcontrol-&gt;private_data. The
        kcontrol-&gt;private_data field is 
        given as the argument of <function>snd_ctl_new()</function>
        (see the later subsection
        <link linkend="control-interface-constructor"><citetitle>Constructor</citetitle></link>).
        </para>
        <para>
 	The <structfield>value</structfield> field is depending on
        the type of control as well as on info callback.  For example,
 	the sb driver uses this field to store the register offset,
        the bit-shift and the bit-mask.  The
        <structfield>private_value</structfield> is set like
          <informalexample>
            <programlisting>
 <![CDATA[
  .private_value = reg | (shift << 16) | (mask << 24)
 ]]>
            </programlisting>
          </informalexample>
 	and is retrieved in callbacks like
          <informalexample>
            <programlisting>
 <![CDATA[
  static int snd_sbmixer_get_single(struct snd_kcontrol *kcontrol,
                                    struct snd_ctl_elem_value *ucontrol)
  {
          int reg = kcontrol->private_value & 0xff;
          int shift = (kcontrol->private_value >> 16) & 0xff;
          int mask = (kcontrol->private_value >> 24) & 0xff;
          ....
  }
 ]]>
            </programlisting>
          </informalexample>
 	</para>
 	<para>
 	In <structfield>get</structfield> callback, you have to fill all the elements if the
        control has more than one elements,
        i.e. <structfield>count</structfield> &gt; 1.
 	In the example above, we filled only one element
        (<structfield>value.integer.value[0]</structfield>) since it's
        assumed as <structfield>count</structfield> = 1.
        </para>
      </section>
      <section id="control-interface-callbacks-put">
        <title>put callback</title>
        <para>
          This callback is used to write a value from the user-space.
        </para>
        <para>
          For example,
          <example>
 	    <title>Example of put callback</title>
            <programlisting>
 <![CDATA[
  static int snd_myctl_put(struct snd_kcontrol *kcontrol,
                           struct snd_ctl_elem_value *ucontrol)
  {
          struct mychip *chip = snd_kcontrol_chip(kcontrol);
          int changed = 0;
          if (chip->current_value !=
               ucontrol->value.integer.value[0]) {
                  change_current_value(chip,
                              ucontrol->value.integer.value[0]);
                  changed = 1;
          }
          return changed;
  }
 ]]>
            </programlisting>
          </example>
          As seen above, you have to return 1 if the value is
        changed. If the value is not changed, return 0 instead. 
 	If any fatal error happens, return a negative error code as
        usual.
        </para>
        <para>
 	Like <structfield>get</structfield> callback,
 	when the control has more than one elements,
 	all elemehts must be evaluated in this callback, too.
        </para>
      </section>
      <section id="control-interface-callbacks-all">
        <title>Callbacks are not atomic</title>
        <para>
          All these three callbacks are basically not atomic.
        </para>
      </section>
    </section>
    <section id="control-interface-constructor">
      <title>Constructor</title>
      <para>
        When everything is ready, finally we can create a new
      control. For creating a control, there are two functions to be
      called, <function>snd_ctl_new1()</function> and
      <function>snd_ctl_add()</function>. 
      </para>
      <para>
        In the simplest way, you can do like this:
        <informalexample>
          <programlisting>
 <![CDATA[
  if ((err = snd_ctl_add(card, snd_ctl_new1(&my_control, chip))) < 0)
          return err;
 ]]>
          </programlisting>
        </informalexample>
        where <parameter>my_control</parameter> is the
      struct <structname>snd_kcontrol_new</structname> object defined above, and chip
      is the object pointer to be passed to
      kcontrol-&gt;private_data 
      which can be referred in callbacks. 
      </para>
      <para>
        <function>snd_ctl_new1()</function> allocates a new
      <structname>snd_kcontrol</structname> instance (that's why the definition
      of <parameter>my_control</parameter> can be with
      <parameter>__devinitdata</parameter> 
      prefix), and <function>snd_ctl_add</function> assigns the given
      control component to the card. 
      </para>
    </section>
    <section id="control-interface-change-notification">
      <title>Change Notification</title>
      <para>
        If you need to change and update a control in the interrupt
      routine, you can call <function>snd_ctl_notify()</function>. For
      example, 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_ctl_notify(card, SNDRV_CTL_EVENT_MASK_VALUE, id_pointer);
 ]]>
          </programlisting>
        </informalexample>
        This function takes the card pointer, the event-mask, and the
      control id pointer for the notification. The event-mask
      specifies the types of notification, for example, in the above
      example, the change of control values is notified.
      The id pointer is the pointer of struct <structname>snd_ctl_elem_id</structname>
      to be notified.
      You can find some examples in <filename>es1938.c</filename> or
      <filename>es1968.c</filename> for hardware volume interrupts. 
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- API for AC97 Codec  -->
 <!-- ****************************************************** -->
  <chapter id="api-ac97">
    <title>API for AC97 Codec</title>
    <section>
      <title>General</title>
      <para>
        The ALSA AC97 codec layer is a well-defined one, and you don't
      have to write many codes to control it. Only low-level control
      routines are necessary. The AC97 codec API is defined in
      <filename>&lt;sound/ac97_codec.h&gt;</filename>. 
      </para>
    </section>
    <section id="api-ac97-example">
      <title>Full Code Example</title>
      <para>
          <example>
 	    <title>Example of AC97 Interface</title>
            <programlisting>
 <![CDATA[
  struct mychip {
          ....
          struct snd_ac97 *ac97;
          ....
  };
  static unsigned short snd_mychip_ac97_read(struct snd_ac97 *ac97,
                                             unsigned short reg)
  {
          struct mychip *chip = ac97->private_data;
          ....
          // read a register value here from the codec
          return the_register_value;
  }
  static void snd_mychip_ac97_write(struct snd_ac97 *ac97,
                                   unsigned short reg, unsigned short val)
  {
          struct mychip *chip = ac97->private_data;
          ....
          // write the given register value to the codec
  }
  static int snd_mychip_ac97(struct mychip *chip)
  {
          struct snd_ac97_bus *bus;
          struct snd_ac97_template ac97;
          int err;
          static struct snd_ac97_bus_ops ops = {
                  .write = snd_mychip_ac97_write,
                  .read = snd_mychip_ac97_read,
          };
          if ((err = snd_ac97_bus(chip->card, 0, &ops, NULL, &bus)) < 0)
                  return err;
          memset(&ac97, 0, sizeof(ac97));
          ac97.private_data = chip;
          return snd_ac97_mixer(bus, &ac97, &chip->ac97);
  }
 ]]>
          </programlisting>
        </example>
      </para>
    </section>
    <section id="api-ac97-constructor">
      <title>Constructor</title>
      <para>
        For creating an ac97 instance, first call <function>snd_ac97_bus</function>
      with an <type>ac97_bus_ops_t</type> record with callback functions.
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_ac97_bus *bus;
  static struct snd_ac97_bus_ops ops = {
        .write = snd_mychip_ac97_write,
        .read = snd_mychip_ac97_read,
  };
  snd_ac97_bus(card, 0, &ops, NULL, &pbus);
 ]]>
          </programlisting>
        </informalexample>
      The bus record is shared among all belonging ac97 instances.
      </para>
      <para>
      And then call <function>snd_ac97_mixer()</function> with an
      struct <structname>snd_ac97_template</structname>
      record together with the bus pointer created above.
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_ac97_template ac97;
  int err;
  memset(&ac97, 0, sizeof(ac97));
  ac97.private_data = chip;
  snd_ac97_mixer(bus, &ac97, &chip->ac97);
 ]]>
          </programlisting>
        </informalexample>
        where chip-&gt;ac97 is the pointer of a newly created
        <type>ac97_t</type> instance.
        In this case, the chip pointer is set as the private data, so that
        the read/write callback functions can refer to this chip instance.
        This instance is not necessarily stored in the chip
 	record.  When you need to change the register values from the
        driver, or need the suspend/resume of ac97 codecs, keep this
        pointer to pass to the corresponding functions.
      </para>
    </section>
    <section id="api-ac97-callbacks">
      <title>Callbacks</title>
      <para>
        The standard callbacks are <structfield>read</structfield> and
      <structfield>write</structfield>. Obviously they 
      correspond to the functions for read and write accesses to the
      hardware low-level codes. 
      </para>
      <para>
        The <structfield>read</structfield> callback returns the
        register value specified in the argument. 
        <informalexample>
          <programlisting>
 <![CDATA[
  static unsigned short snd_mychip_ac97_read(struct snd_ac97 *ac97,
                                             unsigned short reg)
  {
          struct mychip *chip = ac97->private_data;
          ....
          return the_register_value;
  }
 ]]>
          </programlisting>
        </informalexample>
        Here, the chip can be cast from ac97-&gt;private_data.
      </para>
      <para>
        Meanwhile, the <structfield>write</structfield> callback is
        used to set the register value. 
        <informalexample>
          <programlisting>
 <![CDATA[
  static void snd_mychip_ac97_write(struct snd_ac97 *ac97,
                       unsigned short reg, unsigned short val)
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
      These callbacks are non-atomic like the callbacks of control API.
      </para>
      <para>
        There are also other callbacks:
      <structfield>reset</structfield>,
      <structfield>wait</structfield> and
      <structfield>init</structfield>. 
      </para>
      <para>
        The <structfield>reset</structfield> callback is used to reset
      the codec. If the chip requires a special way of reset, you can
      define this callback. 
      </para>
      <para>
        The <structfield>wait</structfield> callback is used for a
      certain wait at the standard initialization of the codec. If the
      chip requires the extra wait-time, define this callback. 
      </para>
      <para>
        The <structfield>init</structfield> callback is used for
      additional initialization of the codec.
      </para>
    </section>
    <section id="api-ac97-updating-registers">
      <title>Updating Registers in The Driver</title>
      <para>
        If you need to access to the codec from the driver, you can
      call the following functions:
      <function>snd_ac97_write()</function>,
      <function>snd_ac97_read()</function>,
      <function>snd_ac97_update()</function> and
      <function>snd_ac97_update_bits()</function>. 
      </para>
      <para>
        Both <function>snd_ac97_write()</function> and
        <function>snd_ac97_update()</function> functions are used to
        set a value to the given register
        (<constant>AC97_XXX</constant>). The difference between them is
        that <function>snd_ac97_update()</function> doesn't write a
        value if the given value has been already set, while
        <function>snd_ac97_write()</function> always rewrites the
        value. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_ac97_write(ac97, AC97_MASTER, 0x8080);
  snd_ac97_update(ac97, AC97_MASTER, 0x8080);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        <function>snd_ac97_read()</function> is used to read the value
        of the given register. For example, 
        <informalexample>
          <programlisting>
 <![CDATA[
  value = snd_ac97_read(ac97, AC97_MASTER);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        <function>snd_ac97_update_bits()</function> is used to update
        some bits of the given register.  
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_ac97_update_bits(ac97, reg, mask, value);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        Also, there is a function to change the sample rate (of a
        certain register such as
        <constant>AC97_PCM_FRONT_DAC_RATE</constant>) when VRA or
        DRA is supported by the codec:
        <function>snd_ac97_set_rate()</function>. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_ac97_set_rate(ac97, AC97_PCM_FRONT_DAC_RATE, 44100);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The following registers are available for setting the rate:
      <constant>AC97_PCM_MIC_ADC_RATE</constant>,
      <constant>AC97_PCM_FRONT_DAC_RATE</constant>,
      <constant>AC97_PCM_LR_ADC_RATE</constant>,
      <constant>AC97_SPDIF</constant>. When the
      <constant>AC97_SPDIF</constant> is specified, the register is
      not really changed but the corresponding IEC958 status bits will
      be updated. 
      </para>
    </section>
    <section id="api-ac97-clock-adjustment">
      <title>Clock Adjustment</title>
      <para>
        On some chip, the clock of the codec isn't 48000 but using a
      PCI clock (to save a quartz!). In this case, change the field
      bus-&gt;clock to the corresponding
      value. For example, intel8x0 
      and es1968 drivers have the auto-measurement function of the
      clock. 
      </para>
    </section>
    <section id="api-ac97-proc-files">
      <title>Proc Files</title>
      <para>
        The ALSA AC97 interface will create a proc file such as
      <filename>/proc/asound/card0/codec97#0/ac97#0-0</filename> and
      <filename>ac97#0-0+regs</filename>. You can refer to these files to
      see the current status and registers of the codec. 
      </para>
    </section>
    <section id="api-ac97-multiple-codecs">
      <title>Multiple Codecs</title>
      <para>
        When there are several codecs on the same card, you need to
      call <function>snd_ac97_mixer()</function> multiple times with
      ac97.num=1 or greater. The <structfield>num</structfield> field
      specifies the codec 
      number. 
      </para>
      <para>
        If you have set up multiple codecs, you need to either write
      different callbacks for each codec or check
      ac97-&gt;num in the 
      callback routines. 
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- MIDI (MPU401-UART) Interface  -->
 <!-- ****************************************************** -->
  <chapter id="midi-interface">
    <title>MIDI (MPU401-UART) Interface</title>
    <section id="midi-interface-general">
      <title>General</title>
      <para>
        Many soundcards have built-in MIDI (MPU401-UART)
      interfaces. When the soundcard supports the standard MPU401-UART
      interface, most likely you can use the ALSA MPU401-UART API. The
      MPU401-UART API is defined in
      <filename>&lt;sound/mpu401.h&gt;</filename>. 
      </para>
      <para>
        Some soundchips have similar but a little bit different
      implementation of mpu401 stuff. For example, emu10k1 has its own
      mpu401 routines. 
      </para>
    </section>
    <section id="midi-interface-constructor">
      <title>Constructor</title>
      <para>
        For creating a rawmidi object, call
      <function>snd_mpu401_uart_new()</function>. 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_rawmidi *rmidi;
  snd_mpu401_uart_new(card, 0, MPU401_HW_MPU401, port, info_flags,
                      irq, irq_flags, &rmidi);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The first argument is the card pointer, and the second is the
      index of this component. You can create up to 8 rawmidi
      devices. 
      </para>
      <para>
        The third argument is the type of the hardware,
      <constant>MPU401_HW_XXX</constant>. If it's not a special one,
      you can use <constant>MPU401_HW_MPU401</constant>. 
      </para>
      <para>
        The 4th argument is the i/o port address. Many
      backward-compatible MPU401 has an i/o port such as 0x330. Or, it
      might be a part of its own PCI i/o region. It depends on the
      chip design. 
      </para>
      <para>
 	The 5th argument is bitflags for additional information.
        When the i/o port address above is a part of the PCI i/o
      region, the MPU401 i/o port might have been already allocated
      (reserved) by the driver itself. In such a case, pass a bit flag
      <constant>MPU401_INFO_INTEGRATED</constant>,
      and 
      the mpu401-uart layer will allocate the i/o ports by itself. 
      </para>
 	<para>
 	When the controller supports only the input or output MIDI stream,
 	pass <constant>MPU401_INFO_INPUT</constant> or
 	<constant>MPU401_INFO_OUTPUT</constant> bitflag, respectively.
 	Then the rawmidi instance is created as a single stream.
 	</para>
 	<para>
 	<constant>MPU401_INFO_MMIO</constant> bitflag is used to change
 	the access method to MMIO (via readb and writeb) instead of
 	iob and outb.  In this case, you have to pass the iomapped address
 	to <function>snd_mpu401_uart_new()</function>.
 	</para>
 	<para>
 	When <constant>MPU401_INFO_TX_IRQ</constant> is set, the output
 	stream isn't checked in the default interrupt handler.  The driver
 	needs to call <function>snd_mpu401_uart_interrupt_tx()</function>
 	by itself to start processing the output stream in irq handler.
 	</para>
      <para>
        Usually, the port address corresponds to the command port and
        port + 1 corresponds to the data port. If not, you may change
        the <structfield>cport</structfield> field of
        struct <structname>snd_mpu401</structname> manually 
        afterward. However, <structname>snd_mpu401</structname> pointer is not
        returned explicitly by
        <function>snd_mpu401_uart_new()</function>. You need to cast
        rmidi-&gt;private_data to
        <structname>snd_mpu401</structname> explicitly, 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_mpu401 *mpu;
  mpu = rmidi->private_data;
 ]]>
          </programlisting>
        </informalexample>
        and reset the cport as you like:
        <informalexample>
          <programlisting>
 <![CDATA[
  mpu->cport = my_own_control_port;
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The 6th argument specifies the irq number for UART. If the irq
      is already allocated, pass 0 to the 7th argument
      (<parameter>irq_flags</parameter>). Otherwise, pass the flags
      for irq allocation 
      (<constant>SA_XXX</constant> bits) to it, and the irq will be
      reserved by the mpu401-uart layer. If the card doesn't generates
      UART interrupts, pass -1 as the irq number. Then a timer
      interrupt will be invoked for polling. 
      </para>
    </section>
    <section id="midi-interface-interrupt-handler">
      <title>Interrupt Handler</title>
      <para>
        When the interrupt is allocated in
      <function>snd_mpu401_uart_new()</function>, the private
      interrupt handler is used, hence you don't have to do nothing
      else than creating the mpu401 stuff. Otherwise, you have to call
      <function>snd_mpu401_uart_interrupt()</function> explicitly when
      a UART interrupt is invoked and checked in your own interrupt
      handler.  
      </para>
      <para>
        In this case, you need to pass the private_data of the
        returned rawmidi object from
        <function>snd_mpu401_uart_new()</function> as the second
        argument of <function>snd_mpu401_uart_interrupt()</function>. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_mpu401_uart_interrupt(irq, rmidi->private_data, regs);
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- RawMIDI Interface  -->
 <!-- ****************************************************** -->
  <chapter id="rawmidi-interface">
    <title>RawMIDI Interface</title>
    <section id="rawmidi-interface-overview">
      <title>Overview</title>
      <para>
      The raw MIDI interface is used for hardware MIDI ports that can
      be accessed as a byte stream.  It is not used for synthesizer
      chips that do not directly understand MIDI.
      </para>
      <para>
      ALSA handles file and buffer management.  All you have to do is
      to write some code to move data between the buffer and the
      hardware.
      </para>
      <para>
      The rawmidi API is defined in
      <filename>&lt;sound/rawmidi.h&gt;</filename>.
      </para>
    </section>
    <section id="rawmidi-interface-constructor">
      <title>Constructor</title>
      <para>
      To create a rawmidi device, call the
      <function>snd_rawmidi_new</function> function:
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_rawmidi *rmidi;
  err = snd_rawmidi_new(chip->card, "MyMIDI", 0, outs, ins, &rmidi);
  if (err < 0)
          return err;
  rmidi->private_data = chip;
  strcpy(rmidi->name, "My MIDI");
  rmidi->info_flags = SNDRV_RAWMIDI_INFO_OUTPUT |
                      SNDRV_RAWMIDI_INFO_INPUT |
                      SNDRV_RAWMIDI_INFO_DUPLEX;
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
      The first argument is the card pointer, the second argument is
      the ID string.
      </para>
      <para>
      The third argument is the index of this component.  You can
      create up to 8 rawmidi devices.
      </para>
      <para>
      The fourth and fifth arguments are the number of output and
      input substreams, respectively, of this device.  (A substream is
      the equivalent of a MIDI port.)
      </para>
      <para>
      Set the <structfield>info_flags</structfield> field to specify
      the capabilities of the device.
      Set <constant>SNDRV_RAWMIDI_INFO_OUTPUT</constant> if there is
      at least one output port,
      <constant>SNDRV_RAWMIDI_INFO_INPUT</constant> if there is at
      least one input port,
      and <constant>SNDRV_RAWMIDI_INFO_DUPLEX</constant> if the device
      can handle output and input at the same time.
      </para>
      <para>
      After the rawmidi device is created, you need to set the
      operators (callbacks) for each substream.  There are helper
      functions to set the operators for all substream of a device:
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_rawmidi_set_ops(rmidi, SNDRV_RAWMIDI_STREAM_OUTPUT, &snd_mymidi_output_ops);
  snd_rawmidi_set_ops(rmidi, SNDRV_RAWMIDI_STREAM_INPUT, &snd_mymidi_input_ops);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
      The operators are usually defined like this:
        <informalexample>
          <programlisting>
 <![CDATA[
  static struct snd_rawmidi_ops snd_mymidi_output_ops = {
          .open =    snd_mymidi_output_open,
          .close =   snd_mymidi_output_close,
          .trigger = snd_mymidi_output_trigger,
  };
 ]]>
          </programlisting>
        </informalexample>
      These callbacks are explained in the <link
      linkend="rawmidi-interface-callbacks"><citetitle>Callbacks</citetitle></link>
      section.
      </para>
      <para>
      If there is more than one substream, you should give each one a
      unique name:
        <informalexample>
          <programlisting>
 <![CDATA[
  struct list_head *list;
  struct snd_rawmidi_substream *substream;
  list_for_each(list, &rmidi->streams[SNDRV_RAWMIDI_STREAM_OUTPUT].substreams) {
          substream = list_entry(list, struct snd_rawmidi_substream, list);
          sprintf(substream->name, "My MIDI Port %d", substream->number + 1);
  }
  /* same for SNDRV_RAWMIDI_STREAM_INPUT */
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
    <section id="rawmidi-interface-callbacks">
      <title>Callbacks</title>
      <para>
      In all callbacks, the private data that you've set for the
      rawmidi device can be accessed as
      substream-&gt;rmidi-&gt;private_data.
      <!-- <code> isn't available before DocBook 4.3 -->
      </para>
      <para>
      If there is more than one port, your callbacks can determine the
      port index from the struct snd_rawmidi_substream data passed to each
      callback:
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_rawmidi_substream *substream;
  int index = substream->number;
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <section id="rawmidi-interface-op-open">
      <title><function>open</function> callback</title>
        <informalexample>
          <programlisting>
 <![CDATA[
  static int snd_xxx_open(struct snd_rawmidi_substream *substream);
 ]]>
          </programlisting>
        </informalexample>
        <para>
        This is called when a substream is opened.
        You can initialize the hardware here, but you should not yet
        start transmitting/receiving data.
        </para>
      </section>
      <section id="rawmidi-interface-op-close">
      <title><function>close</function> callback</title>
        <informalexample>
          <programlisting>
 <![CDATA[
  static int snd_xxx_close(struct snd_rawmidi_substream *substream);
 ]]>
          </programlisting>
        </informalexample>
        <para>
        Guess what.
        </para>
        <para>
        The <function>open</function> and <function>close</function>
        callbacks of a rawmidi device are serialized with a mutex,
        and can sleep.
        </para>
      </section>
      <section id="rawmidi-interface-op-trigger-out">
      <title><function>trigger</function> callback for output
      substreams</title>
        <informalexample>
          <programlisting>
 <![CDATA[
  static void snd_xxx_output_trigger(struct snd_rawmidi_substream *substream, int up);
 ]]>
          </programlisting>
        </informalexample>
        <para>
        This is called with a nonzero <parameter>up</parameter>
        parameter when there is some data in the substream buffer that
        must be transmitted.
        </para>
        <para>
        To read data from the buffer, call
        <function>snd_rawmidi_transmit_peek</function>.  It will
        return the number of bytes that have been read; this will be
        less than the number of bytes requested when there is no more
        data in the buffer.
        After the data has been transmitted successfully, call
        <function>snd_rawmidi_transmit_ack</function> to remove the
        data from the substream buffer:
          <informalexample>
            <programlisting>
 <![CDATA[
  unsigned char data;
  while (snd_rawmidi_transmit_peek(substream, &data, 1) == 1) {
          if (snd_mychip_try_to_transmit(data))
                  snd_rawmidi_transmit_ack(substream, 1);
          else
                  break; /* hardware FIFO full */
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
        If you know beforehand that the hardware will accept data, you
        can use the <function>snd_rawmidi_transmit</function> function
        which reads some data and removes it from the buffer at once:
          <informalexample>
            <programlisting>
 <![CDATA[
  while (snd_mychip_transmit_possible()) {
          unsigned char data;
          if (snd_rawmidi_transmit(substream, &data, 1) != 1)
                  break; /* no more data */
          snd_mychip_transmit(data);
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
        <para>
        If you know beforehand how many bytes you can accept, you can
        use a buffer size greater than one with the
        <function>snd_rawmidi_transmit*</function> functions.
        </para>
        <para>
        The <function>trigger</function> callback must not sleep.  If
        the hardware FIFO is full before the substream buffer has been
        emptied, you have to continue transmitting data later, either
        in an interrupt handler, or with a timer if the hardware
        doesn't have a MIDI transmit interrupt.
        </para>
        <para>
        The <function>trigger</function> callback is called with a
        zero <parameter>up</parameter> parameter when the transmission
        of data should be aborted.
        </para>
      </section>
      <section id="rawmidi-interface-op-trigger-in">
      <title><function>trigger</function> callback for input
      substreams</title>
        <informalexample>
          <programlisting>
 <![CDATA[
  static void snd_xxx_input_trigger(struct snd_rawmidi_substream *substream, int up);
 ]]>
          </programlisting>
        </informalexample>
        <para>
        This is called with a nonzero <parameter>up</parameter>
        parameter to enable receiving data, or with a zero
        <parameter>up</parameter> parameter do disable receiving data.
        </para>
        <para>
        The <function>trigger</function> callback must not sleep; the
        actual reading of data from the device is usually done in an
        interrupt handler.
        </para>
        <para>
        When data reception is enabled, your interrupt handler should
        call <function>snd_rawmidi_receive</function> for all received
        data:
          <informalexample>
            <programlisting>
 <![CDATA[
  void snd_mychip_midi_interrupt(...)
  {
          while (mychip_midi_available()) {
                  unsigned char data;
                  data = mychip_midi_read();
                  snd_rawmidi_receive(substream, &data, 1);
          }
  }
 ]]>
            </programlisting>
          </informalexample>
        </para>
      </section>
      <section id="rawmidi-interface-op-drain">
      <title><function>drain</function> callback</title>
        <informalexample>
          <programlisting>
 <![CDATA[
  static void snd_xxx_drain(struct snd_rawmidi_substream *substream);
 ]]>
          </programlisting>
        </informalexample>
        <para>
        This is only used with output substreams.  This function should wait
        until all data read from the substream buffer has been transmitted.
        This ensures that the device can be closed and the driver unloaded
        without losing data.
        </para>
        <para>
        This callback is optional.  If you do not set
        <structfield>drain</structfield> in the struct snd_rawmidi_ops
        structure, ALSA will simply wait for 50&nbsp;milliseconds
        instead.
        </para>
      </section>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Miscellaneous Devices  -->
 <!-- ****************************************************** -->
  <chapter id="misc-devices">
    <title>Miscellaneous Devices</title>
    <section id="misc-devices-opl3">
      <title>FM OPL3</title>
      <para>
        The FM OPL3 is still used on many chips (mainly for backward
      compatibility). ALSA has a nice OPL3 FM control layer, too. The
      OPL3 API is defined in
      <filename>&lt;sound/opl3.h&gt;</filename>. 
      </para>
      <para>
        FM registers can be directly accessed through direct-FM API,
      defined in <filename>&lt;sound/asound_fm.h&gt;</filename>. In
      ALSA native mode, FM registers are accessed through
      Hardware-Dependant Device direct-FM extension API, whereas in
      OSS compatible mode, FM registers can be accessed with OSS
      direct-FM compatible API on <filename>/dev/dmfmX</filename> device. 
      </para>
      <para>
        For creating the OPL3 component, you have two functions to
        call. The first one is a constructor for <type>opl3_t</type>
        instance. 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_opl3 *opl3;
  snd_opl3_create(card, lport, rport, OPL3_HW_OPL3_XXX,
                  integrated, &opl3);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The first argument is the card pointer, the second one is the
      left port address, and the third is the right port address. In
      most cases, the right port is placed at the left port + 2. 
      </para>
      <para>
        The fourth argument is the hardware type.
      </para>
      <para>
        When the left and right ports have been already allocated by
      the card driver, pass non-zero to the fifth argument
      (<parameter>integrated</parameter>). Otherwise, opl3 module will
      allocate the specified ports by itself. 
      </para>
      <para>
        When the accessing to the hardware requires special method
        instead of the standard I/O access, you can create opl3 instance
        separately with <function>snd_opl3_new()</function>.
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_opl3 *opl3;
  snd_opl3_new(card, OPL3_HW_OPL3_XXX, &opl3);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
 	Then set <structfield>command</structfield>,
 	<structfield>private_data</structfield> and
 	<structfield>private_free</structfield> for the private
 	access function, the private data and the destructor.
 	The l_port and r_port are not necessarily set.  Only the
 	command must be set properly.  You can retrieve the data
 	from opl3-&gt;private_data field.
      </para>
      <para>
 	After creating the opl3 instance via <function>snd_opl3_new()</function>,
 	call <function>snd_opl3_init()</function> to initialize the chip to the
 	proper state.  Note that <function>snd_opl3_create()</function> always
 	calls it internally.
      </para>
      <para>
        If the opl3 instance is created successfully, then create a
        hwdep device for this opl3. 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_hwdep *opl3hwdep;
  snd_opl3_hwdep_new(opl3, 0, 1, &opl3hwdep);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The first argument is the <type>opl3_t</type> instance you
      created, and the second is the index number, usually 0. 
      </para>
      <para>
        The third argument is the index-offset for the sequencer
      client assigned to the OPL3 port. When there is an MPU401-UART,
      give 1 for here (UART always takes 0). 
      </para>
    </section>
    <section id="misc-devices-hardware-dependent">
      <title>Hardware-Dependent Devices</title>
      <para>
        Some chips need the access from the user-space for special
      controls or for loading the micro code. In such a case, you can
      create a hwdep (hardware-dependent) device. The hwdep API is
      defined in <filename>&lt;sound/hwdep.h&gt;</filename>. You can
      find examples in opl3 driver or
      <filename>isa/sb/sb16_csp.c</filename>. 
      </para>
      <para>
        Creation of the <type>hwdep</type> instance is done via
        <function>snd_hwdep_new()</function>. 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_hwdep *hw;
  snd_hwdep_new(card, "My HWDEP", 0, &hw);
 ]]>
          </programlisting>
        </informalexample>
        where the third argument is the index number.
      </para>
      <para>
        You can then pass any pointer value to the
        <parameter>private_data</parameter>.
        If you assign a private data, you should define the
        destructor, too. The destructor function is set to
        <structfield>private_free</structfield> field.  
        <informalexample>
          <programlisting>
 <![CDATA[
  struct mydata *p = kmalloc(sizeof(*p), GFP_KERNEL);
  hw->private_data = p;
  hw->private_free = mydata_free;
 ]]>
          </programlisting>
        </informalexample>
        and the implementation of destructor would be:
        <informalexample>
          <programlisting>
 <![CDATA[
  static void mydata_free(struct snd_hwdep *hw)
  {
          struct mydata *p = hw->private_data;
          kfree(p);
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The arbitrary file operations can be defined for this
        instance. The file operators are defined in
        <parameter>ops</parameter> table. For example, assume that
        this chip needs an ioctl. 
        <informalexample>
          <programlisting>
 <![CDATA[
  hw->ops.open = mydata_open;
  hw->ops.ioctl = mydata_ioctl;
  hw->ops.release = mydata_release;
 ]]>
          </programlisting>
        </informalexample>
        And implement the callback functions as you like.
      </para>
    </section>
    <section id="misc-devices-IEC958">
      <title>IEC958 (S/PDIF)</title>
      <para>
        Usually the controls for IEC958 devices are implemented via
      control interface. There is a macro to compose a name string for
      IEC958 controls, <function>SNDRV_CTL_NAME_IEC958()</function>
      defined in <filename>&lt;include/asound.h&gt;</filename>.  
      </para>
      <para>
        There are some standard controls for IEC958 status bits. These
      controls use the type <type>SNDRV_CTL_ELEM_TYPE_IEC958</type>,
      and the size of element is fixed as 4 bytes array
      (value.iec958.status[x]). For <structfield>info</structfield>
      callback, you don't specify 
      the value field for this type (the count field must be set,
      though). 
      </para>
      <para>
        <quote>IEC958 Playback Con Mask</quote> is used to return the
      bit-mask for the IEC958 status bits of consumer mode. Similarly,
      <quote>IEC958 Playback Pro Mask</quote> returns the bitmask for
      professional mode. They are read-only controls, and are defined
      as MIXER controls (iface =
      <constant>SNDRV_CTL_ELEM_IFACE_MIXER</constant>).  
      </para>
      <para>
        Meanwhile, <quote>IEC958 Playback Default</quote> control is
      defined for getting and setting the current default IEC958
      bits. Note that this one is usually defined as a PCM control
      (iface = <constant>SNDRV_CTL_ELEM_IFACE_PCM</constant>),
      although in some places it's defined as a MIXER control. 
      </para>
      <para>
        In addition, you can define the control switches to
      enable/disable or to set the raw bit mode. The implementation
      will depend on the chip, but the control should be named as
      <quote>IEC958 xxx</quote>, preferably using
      <function>SNDRV_CTL_NAME_IEC958()</function> macro. 
      </para>
      <para>
        You can find several cases, for example,
      <filename>pci/emu10k1</filename>,
      <filename>pci/ice1712</filename>, or
      <filename>pci/cmipci.c</filename>.  
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Buffer and Memory Management  -->
 <!-- ****************************************************** -->
  <chapter id="buffer-and-memory">
    <title>Buffer and Memory Management</title>
    <section id="buffer-and-memory-buffer-types">
      <title>Buffer Types</title>
      <para>
        ALSA provides several different buffer allocation functions
      depending on the bus and the architecture. All these have a
      consistent API. The allocation of physically-contiguous pages is
      done via 
      <function>snd_malloc_xxx_pages()</function> function, where xxx
      is the bus type. 
      </para>
      <para>
        The allocation of pages with fallback is
      <function>snd_malloc_xxx_pages_fallback()</function>. This
      function tries to allocate the specified pages but if the pages
      are not available, it tries to reduce the page sizes until the
      enough space is found.
      </para>
      <para>
      For releasing the space, call
      <function>snd_free_xxx_pages()</function> function. 
      </para>
      <para>
      Usually, ALSA drivers try to allocate and reserve
       a large contiguous physical space
       at the time the module is loaded for the later use.
       This is called <quote>pre-allocation</quote>.
       As already written, you can call the following function at the
       construction of pcm instance (in the case of PCI bus). 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_pcm_lib_preallocate_pages_for_all(pcm, SNDRV_DMA_TYPE_DEV,
                                        snd_dma_pci_data(pci), size, max);
 ]]>
          </programlisting>
        </informalexample>
        where <parameter>size</parameter> is the byte size to be
      pre-allocated and the <parameter>max</parameter> is the maximal
      size to be changed via <filename>prealloc</filename> proc file.
      The allocator will try to get as large area as possible
      within the given size. 
      </para>
      <para>
      The second argument (type) and the third argument (device pointer)
      are dependent on the bus.
      In the case of ISA bus, pass <function>snd_dma_isa_data()</function>
      as the third argument with <constant>SNDRV_DMA_TYPE_DEV</constant> type.
      For the continuous buffer unrelated to the bus can be pre-allocated
      with <constant>SNDRV_DMA_TYPE_CONTINUOUS</constant> type and the
      <function>snd_dma_continuous_data(GFP_KERNEL)</function> device pointer,
      whereh <constant>GFP_KERNEL</constant> is the kernel allocation flag to
      use.  For the SBUS, <constant>SNDRV_DMA_TYPE_SBUS</constant> and
      <function>snd_dma_sbus_data(sbus_dev)</function> are used instead.
      For the PCI scatter-gather buffers, use
      <constant>SNDRV_DMA_TYPE_DEV_SG</constant> with
      <function>snd_dma_pci_data(pci)</function>
      (see the section
          <link linkend="buffer-and-memory-non-contiguous"><citetitle>Non-Contiguous Buffers
          </citetitle></link>).
      </para>
      <para>
        Once when the buffer is pre-allocated, you can use the
        allocator in the <structfield>hw_params</structfield> callback 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_pcm_lib_malloc_pages(substream, size);
 ]]>
          </programlisting>
        </informalexample>
        Note that you have to pre-allocate to use this function.
      </para>
    </section>
    <section id="buffer-and-memory-external-hardware">
      <title>External Hardware Buffers</title>
      <para>
        Some chips have their own hardware buffers and the DMA
      transfer from the host memory is not available. In such a case,
      you need to either 1) copy/set the audio data directly to the
      external hardware buffer, or 2) make an intermediate buffer and
      copy/set the data from it to the external hardware buffer in
      interrupts (or in tasklets, preferably).
      </para>
      <para>
        The first case works fine if the external hardware buffer is enough
      large.  This method doesn't need any extra buffers and thus is
      more effective. You need to define the
      <structfield>copy</structfield> and
      <structfield>silence</structfield> callbacks for 
      the data transfer. However, there is a drawback: it cannot
      be mmapped. The examples are GUS's GF1 PCM or emu8000's
      wavetable PCM. 
      </para>
      <para>
        The second case allows the mmap of the buffer, although you have
      to handle an interrupt or a tasklet for transferring the data
      from the intermediate buffer to the hardware buffer. You can find an
      example in vxpocket driver. 
      </para>
      <para>
        Another case is that the chip uses a PCI memory-map
      region for the buffer instead of the host memory. In this case,
      mmap is available only on certain architectures like intel. In
      non-mmap mode, the data cannot be transferred as the normal
      way. Thus you need to define <structfield>copy</structfield> and
      <structfield>silence</structfield> callbacks as well 
      as in the cases above. The examples are found in
      <filename>rme32.c</filename> and <filename>rme96.c</filename>. 
      </para>
      <para>
        The implementation of <structfield>copy</structfield> and
        <structfield>silence</structfield> callbacks depends upon 
        whether the hardware supports interleaved or non-interleaved
        samples. The <structfield>copy</structfield> callback is
        defined like below, a bit 
        differently depending whether the direction is playback or
        capture: 
        <informalexample>
          <programlisting>
 <![CDATA[
  static int playback_copy(struct snd_pcm_substream *substream, int channel,
               snd_pcm_uframes_t pos, void *src, snd_pcm_uframes_t count);
  static int capture_copy(struct snd_pcm_substream *substream, int channel,
               snd_pcm_uframes_t pos, void *dst, snd_pcm_uframes_t count);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        In the case of interleaved samples, the second argument
      (<parameter>channel</parameter>) is not used. The third argument
      (<parameter>pos</parameter>) points the 
      current position offset in frames. 
      </para>
      <para>
        The meaning of the fourth argument is different between
      playback and capture. For playback, it holds the source data
      pointer, and for capture, it's the destination data pointer. 
      </para>
      <para>
        The last argument is the number of frames to be copied.
      </para>
      <para>
        What you have to do in this callback is again different
        between playback and capture directions. In the case of
        playback, you do: copy the given amount of data
        (<parameter>count</parameter>) at the specified pointer
        (<parameter>src</parameter>) to the specified offset
        (<parameter>pos</parameter>) on the hardware buffer. When
        coded like memcpy-like way, the copy would be like: 
        <informalexample>
          <programlisting>
 <![CDATA[
  my_memcpy(my_buffer + frames_to_bytes(runtime, pos), src,
            frames_to_bytes(runtime, count));
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        For the capture direction, you do: copy the given amount of
        data (<parameter>count</parameter>) at the specified offset
        (<parameter>pos</parameter>) on the hardware buffer to the
        specified pointer (<parameter>dst</parameter>). 
        <informalexample>
          <programlisting>
 <![CDATA[
  my_memcpy(dst, my_buffer + frames_to_bytes(runtime, pos),
            frames_to_bytes(runtime, count));
 ]]>
          </programlisting>
        </informalexample>
        Note that both of the position and the data amount are given
      in frames. 
      </para>
      <para>
        In the case of non-interleaved samples, the implementation
      will be a bit more complicated. 
      </para>
      <para>
        You need to check the channel argument, and if it's -1, copy
      the whole channels. Otherwise, you have to copy only the
      specified channel. Please check
      <filename>isa/gus/gus_pcm.c</filename> as an example. 
      </para>
      <para>
        The <structfield>silence</structfield> callback is also
        implemented in a similar way. 
        <informalexample>
          <programlisting>
 <![CDATA[
  static int silence(struct snd_pcm_substream *substream, int channel,
                     snd_pcm_uframes_t pos, snd_pcm_uframes_t count);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The meanings of arguments are identical with the
      <structfield>copy</structfield> 
      callback, although there is no <parameter>src/dst</parameter>
      argument. In the case of interleaved samples, the channel
      argument has no meaning, as well as on
      <structfield>copy</structfield> callback.  
      </para>
      <para>
        The role of <structfield>silence</structfield> callback is to
        set the given amount 
        (<parameter>count</parameter>) of silence data at the
        specified offset (<parameter>pos</parameter>) on the hardware
        buffer. Suppose that the data format is signed (that is, the
        silent-data is 0), and the implementation using a memset-like
        function would be like: 
        <informalexample>
          <programlisting>
 <![CDATA[
  my_memcpy(my_buffer + frames_to_bytes(runtime, pos), 0,
            frames_to_bytes(runtime, count));
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        In the case of non-interleaved samples, again, the
      implementation becomes a bit more complicated. See, for example,
      <filename>isa/gus/gus_pcm.c</filename>. 
      </para>
    </section>
    <section id="buffer-and-memory-non-contiguous">
      <title>Non-Contiguous Buffers</title>
      <para>
        If your hardware supports the page table like emu10k1 or the
      buffer descriptors like via82xx, you can use the scatter-gather
      (SG) DMA. ALSA provides an interface for handling SG-buffers.
      The API is provided in <filename>&lt;sound/pcm.h&gt;</filename>. 
      </para>
      <para>
        For creating the SG-buffer handler, call
        <function>snd_pcm_lib_preallocate_pages()</function> or
        <function>snd_pcm_lib_preallocate_pages_for_all()</function>
        with <constant>SNDRV_DMA_TYPE_DEV_SG</constant>
 	in the PCM constructor like other PCI pre-allocator.
        You need to pass the <function>snd_dma_pci_data(pci)</function>,
        where pci is the struct <structname>pci_dev</structname> pointer
        of the chip as well.
        The <type>struct snd_sg_buf</type> instance is created as
        substream-&gt;dma_private. You can cast
        the pointer like: 
        <informalexample>
          <programlisting>
 <![CDATA[
  struct snd_sg_buf *sgbuf = (struct snd_sg_buf *)substream->dma_private;
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        Then call <function>snd_pcm_lib_malloc_pages()</function>
      in <structfield>hw_params</structfield> callback
      as well as in the case of normal PCI buffer.
      The SG-buffer handler will allocate the non-contiguous kernel
      pages of the given size and map them onto the virtually contiguous
      memory.  The virtual pointer is addressed in runtime-&gt;dma_area.
      The physical address (runtime-&gt;dma_addr) is set to zero,
      because the buffer is physically non-contigous.
      The physical address table is set up in sgbuf-&gt;table.
      You can get the physical address at a certain offset via
      <function>snd_pcm_sgbuf_get_addr()</function>. 
      </para>
      <para>
        When a SG-handler is used, you need to set
      <function>snd_pcm_sgbuf_ops_page</function> as
      the <structfield>page</structfield> callback.
      (See <link linkend="pcm-interface-operators-page-callback">
      <citetitle>page callback section</citetitle></link>.)
      </para>
      <para>
        For releasing the data, call
      <function>snd_pcm_lib_free_pages()</function> in the
      <structfield>hw_free</structfield> callback as usual.
      </para>
    </section>
    <section id="buffer-and-memory-vmalloced">
      <title>Vmalloc'ed Buffers</title>
      <para>
        It's possible to use a buffer allocated via
      <function>vmalloc</function>, for example, for an intermediate
      buffer. Since the allocated pages are not contiguous, you need
      to set the <structfield>page</structfield> callback to obtain
      the physical address at every offset. 
      </para>
      <para>
        The implementation of <structfield>page</structfield> callback
        would be like this: 
        <informalexample>
          <programlisting>
 <![CDATA[
  #include <linux/vmalloc.h>
  /* get the physical page pointer on the given offset */
  static struct page *mychip_page(struct snd_pcm_substream *substream,
                                  unsigned long offset)
  {
          void *pageptr = substream->runtime->dma_area + offset;
          return vmalloc_to_page(pageptr);
  }
 ]]>
          </programlisting>
        </informalexample>
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Proc Interface  -->
 <!-- ****************************************************** -->
  <chapter id="proc-interface">
    <title>Proc Interface</title>
    <para>
      ALSA provides an easy interface for procfs. The proc files are
      very useful for debugging. I recommend you set up proc files if
      you write a driver and want to get a running status or register
      dumps. The API is found in
      <filename>&lt;sound/info.h&gt;</filename>. 
    </para>
    <para>
      For creating a proc file, call
      <function>snd_card_proc_new()</function>. 
      <informalexample>
        <programlisting>
 <![CDATA[
  struct snd_info_entry *entry;
  int err = snd_card_proc_new(card, "my-file", &entry);
 ]]>
        </programlisting>
      </informalexample>
      where the second argument specifies the proc-file name to be
    created. The above example will create a file
    <filename>my-file</filename> under the card directory,
    e.g. <filename>/proc/asound/card0/my-file</filename>. 
    </para>
    <para>
    Like other components, the proc entry created via
    <function>snd_card_proc_new()</function> will be registered and
    released automatically in the card registration and release
    functions.
    </para>
    <para>
      When the creation is successful, the function stores a new
    instance at the pointer given in the third argument.
    It is initialized as a text proc file for read only.  For using
    this proc file as a read-only text file as it is, set the read
    callback with a private data via 
     <function>snd_info_set_text_ops()</function>.
      <informalexample>
        <programlisting>
 <![CDATA[
  snd_info_set_text_ops(entry, chip, my_proc_read);
 ]]>
        </programlisting>
      </informalexample>
    where the second argument (<parameter>chip</parameter>) is the
    private data to be used in the callbacks. The third parameter
    specifies the read buffer size and the fourth
    (<parameter>my_proc_read</parameter>) is the callback function, which
    is defined like
      <informalexample>
        <programlisting>
 <![CDATA[
  static void my_proc_read(struct snd_info_entry *entry,
                           struct snd_info_buffer *buffer);
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
    In the read callback, use <function>snd_iprintf()</function> for
    output strings, which works just like normal
    <function>printf()</function>.  For example,
      <informalexample>
        <programlisting>
 <![CDATA[
  static void my_proc_read(struct snd_info_entry *entry,
                           struct snd_info_buffer *buffer)
  {
          struct my_chip *chip = entry->private_data;
          snd_iprintf(buffer, "This is my chip!\n");
          snd_iprintf(buffer, "Port = %ld\n", chip->port);
  }
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
    The file permission can be changed afterwards.  As default, it's
    set as read only for all users.  If you want to add the write
    permission to the user (root as default), set like below:
      <informalexample>
        <programlisting>
 <![CDATA[
 entry->mode = S_IFREG | S_IRUGO | S_IWUSR;
 ]]>
        </programlisting>
      </informalexample>
    and set the write buffer size and the callback
      <informalexample>
        <programlisting>
 <![CDATA[
  entry->c.text.write = my_proc_write;
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      For the write callback, you can use
    <function>snd_info_get_line()</function> to get a text line, and
    <function>snd_info_get_str()</function> to retrieve a string from
    the line. Some examples are found in
    <filename>core/oss/mixer_oss.c</filename>, core/oss/and
    <filename>pcm_oss.c</filename>. 
    </para>
    <para>
      For a raw-data proc-file, set the attributes like the following:
      <informalexample>
        <programlisting>
 <![CDATA[
  static struct snd_info_entry_ops my_file_io_ops = {
          .read = my_file_io_read,
  };
  entry->content = SNDRV_INFO_CONTENT_DATA;
  entry->private_data = chip;
  entry->c.ops = &my_file_io_ops;
  entry->size = 4096;
  entry->mode = S_IFREG | S_IRUGO;
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      The callback is much more complicated than the text-file
      version. You need to use a low-level i/o functions such as
      <function>copy_from/to_user()</function> to transfer the
      data.
      <informalexample>
        <programlisting>
 <![CDATA[
  static long my_file_io_read(struct snd_info_entry *entry,
                              void *file_private_data,
                              struct file *file,
                              char *buf,
                              unsigned long count,
                              unsigned long pos)
  {
          long size = count;
          if (pos + size > local_max_size)
                  size = local_max_size - pos;
          if (copy_to_user(buf, local_data + pos, size))
                  return -EFAULT;
          return size;
  }
 ]]>
        </programlisting>
      </informalexample>
    </para>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Power Management  -->
 <!-- ****************************************************** -->
  <chapter id="power-management">
    <title>Power Management</title>
    <para>
-      If the chip is supposed to work with with suspend/resume
+      If the chip is supposed to work with suspend/resume
      functions, you need to add the power-management codes to the
      driver. The additional codes for the power-management should be
      <function>ifdef</function>'ed with
      <constant>CONFIG_PM</constant>. 
    </para>
 	<para>
 	If the driver supports the suspend/resume
 	<emphasis>fully</emphasis>, that is, the device can be
 	properly resumed to the status at the suspend is called,
 	you can set <constant>SNDRV_PCM_INFO_RESUME</constant> flag
 	to pcm info field.  Usually, this is possible when the
 	registers of ths chip can be safely saved and restored to the
 	RAM.  If this is set, the trigger callback is called with
 	<constant>SNDRV_PCM_TRIGGER_RESUME</constant> after resume
 	callback is finished. 
 	</para>
 	<para>
 	Even if the driver doesn't support PM fully but only the
 	partial suspend/resume is possible, it's still worthy to
 	implement suspend/resume callbacks.  In such a case, applications
 	would reset the status by calling
 	<function>snd_pcm_prepare()</function> and restart the stream
 	appropriately.  Hence, you can define suspend/resume callbacks
 	below but don't set <constant>SNDRV_PCM_INFO_RESUME</constant>
 	info flag to the PCM.
 	</para>
 	<para>
 	Note that the trigger with SUSPEND can be always called when
 	<function>snd_pcm_suspend_all</function> is called,
 	regardless of <constant>SNDRV_PCM_INFO_RESUME</constant> flag.
 	The <constant>RESUME</constant> flag affects only the behavior
 	of <function>snd_pcm_resume()</function>.
 	(Thus, in theory,
 	<constant>SNDRV_PCM_TRIGGER_RESUME</constant> isn't needed
 	to be handled in the trigger callback when no
 	<constant>SNDRV_PCM_INFO_RESUME</constant> flag is set.  But,
 	it's better to keep it for compatibility reason.)
 	</para>
    <para>
      In the earlier version of ALSA drivers, a common
      power-management layer was provided, but it has been removed.
      The driver needs to define the suspend/resume hooks according to
      the bus the device is assigned.  In the case of PCI driver, the
      callbacks look like below:
      <informalexample>
        <programlisting>
 <![CDATA[
  #ifdef CONFIG_PM
  static int snd_my_suspend(struct pci_dev *pci, pm_message_t state)
  {
          .... /* do things for suspsend */
          return 0;
  }
  static int snd_my_resume(struct pci_dev *pci)
  {
          .... /* do things for suspsend */
          return 0;
  }
  #endif
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      The scheme of the real suspend job is as following.
      <orderedlist>
        <listitem><para>Retrieve the card and the chip data.</para></listitem>
        <listitem><para>Call <function>snd_power_change_state()</function> with
 	  <constant>SNDRV_CTL_POWER_D3hot</constant> to change the
 	  power status.</para></listitem>
        <listitem><para>Call <function>snd_pcm_suspend_all()</function> to suspend the running PCM streams.</para></listitem>
 	<listitem><para>If AC97 codecs are used, call
 	<function>snd_ac97_suspend()</function> for each codec.</para></listitem>
        <listitem><para>Save the register values if necessary.</para></listitem>
        <listitem><para>Stop the hardware if necessary.</para></listitem>
        <listitem><para>Disable the PCI device by calling
 	  <function>pci_disable_device()</function>.  Then, call
          <function>pci_save_state()</function> at last.</para></listitem>
      </orderedlist>
    </para>
    <para>
      A typical code would be like:
      <informalexample>
        <programlisting>
 <![CDATA[
  static int mychip_suspend(struct pci_dev *pci, pm_message_t state)
  {
          /* (1) */
          struct snd_card *card = pci_get_drvdata(pci);
          struct mychip *chip = card->private_data;
          /* (2) */
          snd_power_change_state(card, SNDRV_CTL_POWER_D3hot);
          /* (3) */
          snd_pcm_suspend_all(chip->pcm);
          /* (4) */
          snd_ac97_suspend(chip->ac97);
          /* (5) */
          snd_mychip_save_registers(chip);
          /* (6) */
          snd_mychip_stop_hardware(chip);
          /* (7) */
          pci_disable_device(pci);
          pci_save_state(pci);
          return 0;
  }
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
    The scheme of the real resume job is as following.
    <orderedlist>
    <listitem><para>Retrieve the card and the chip data.</para></listitem>
    <listitem><para>Set up PCI.  First, call <function>pci_restore_state()</function>.
    	Then enable the pci device again by calling <function>pci_enable_device()</function>.
 	Call <function>pci_set_master()</function> if necessary, too.</para></listitem>
    <listitem><para>Re-initialize the chip.</para></listitem>
    <listitem><para>Restore the saved registers if necessary.</para></listitem>
    <listitem><para>Resume the mixer, e.g. calling
    <function>snd_ac97_resume()</function>.</para></listitem>
    <listitem><para>Restart the hardware (if any).</para></listitem>
    <listitem><para>Call <function>snd_power_change_state()</function> with
 	<constant>SNDRV_CTL_POWER_D0</constant> to notify the processes.</para></listitem>
    </orderedlist>
    </para>
    <para>
    A typical code would be like:
      <informalexample>
        <programlisting>
 <![CDATA[
  static int mychip_resume(struct pci_dev *pci)
  {
          /* (1) */
          struct snd_card *card = pci_get_drvdata(pci);
          struct mychip *chip = card->private_data;
          /* (2) */
          pci_restore_state(pci);
          pci_enable_device(pci);
          pci_set_master(pci);
          /* (3) */
          snd_mychip_reinit_chip(chip);
          /* (4) */
          snd_mychip_restore_registers(chip);
          /* (5) */
          snd_ac97_resume(chip->ac97);
          /* (6) */
          snd_mychip_restart_chip(chip);
          /* (7) */
          snd_power_change_state(card, SNDRV_CTL_POWER_D0);
          return 0;
  }
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
 	As shown in the above, it's better to save registers after
 	suspending the PCM operations via
 	<function>snd_pcm_suspend_all()</function> or
 	<function>snd_pcm_suspend()</function>.  It means that the PCM
 	streams are already stoppped when the register snapshot is
 	taken.  But, remind that you don't have to restart the PCM
 	stream in the resume callback. It'll be restarted via 
 	trigger call with <constant>SNDRV_PCM_TRIGGER_RESUME</constant>
 	when necessary.
    </para>
    <para>
      OK, we have all callbacks now. Let's set them up. In the
      initialization of the card, make sure that you can get the chip
      data from the card instance, typically via
      <structfield>private_data</structfield> field, in case you
      created the chip data individually.
      <informalexample>
        <programlisting>
 <![CDATA[
  static int __devinit snd_mychip_probe(struct pci_dev *pci,
                               const struct pci_device_id *pci_id)
  {
          ....
          struct snd_card *card;
          struct mychip *chip;
          ....
          card = snd_card_new(index[dev], id[dev], THIS_MODULE, NULL);
          ....
          chip = kzalloc(sizeof(*chip), GFP_KERNEL);
          ....
          card->private_data = chip;
          ....
  }
 ]]>
        </programlisting>
      </informalexample>
 	When you created the chip data with
 	<function>snd_card_new()</function>, it's anyway accessible
 	via <structfield>private_data</structfield> field.
      <informalexample>
        <programlisting>
 <![CDATA[
  static int __devinit snd_mychip_probe(struct pci_dev *pci,
                               const struct pci_device_id *pci_id)
  {
          ....
          struct snd_card *card;
          struct mychip *chip;
          ....
          card = snd_card_new(index[dev], id[dev], THIS_MODULE,
                              sizeof(struct mychip));
          ....
          chip = card->private_data;
          ....
  }
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      If you need a space for saving the registers, allocate the
 	buffer for it here, too, since it would be fatal
    if you cannot allocate a memory in the suspend phase.
    The allocated buffer should be released in the corresponding
    destructor.
    </para>
    <para>
      And next, set suspend/resume callbacks to the pci_driver.
      <informalexample>
        <programlisting>
 <![CDATA[
  static struct pci_driver driver = {
          .name = "My Chip",
          .id_table = snd_my_ids,
          .probe = snd_my_probe,
          .remove = __devexit_p(snd_my_remove),
  #ifdef CONFIG_PM
          .suspend = snd_my_suspend,
          .resume = snd_my_resume,
  #endif
  };
 ]]>
        </programlisting>
      </informalexample>
    </para>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Module Parameters  -->
 <!-- ****************************************************** -->
  <chapter id="module-parameters">
    <title>Module Parameters</title>
    <para>
      There are standard module options for ALSA. At least, each
      module should have <parameter>index</parameter>,
      <parameter>id</parameter> and <parameter>enable</parameter>
      options. 
    </para>
    <para>
      If the module supports multiple cards (usually up to
      8 = <constant>SNDRV_CARDS</constant> cards), they should be
      arrays.  The default initial values are defined already as
      constants for ease of programming:
      <informalexample>
        <programlisting>
 <![CDATA[
  static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
  static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
  static int enable[SNDRV_CARDS] = SNDRV_DEFAULT_ENABLE_PNP;
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      If the module supports only a single card, they could be single
    variables, instead.  <parameter>enable</parameter> option is not
    always necessary in this case, but it wouldn't be so bad to have a
    dummy option for compatibility.
    </para>
    <para>
      The module parameters must be declared with the standard
    <function>module_param()()</function>,
    <function>module_param_array()()</function> and
    <function>MODULE_PARM_DESC()</function> macros.
    </para>
    <para>
      The typical coding would be like below:
      <informalexample>
        <programlisting>
 <![CDATA[
  #define CARD_NAME "My Chip"
  module_param_array(index, int, NULL, 0444);
  MODULE_PARM_DESC(index, "Index value for " CARD_NAME " soundcard.");
  module_param_array(id, charp, NULL, 0444);
  MODULE_PARM_DESC(id, "ID string for " CARD_NAME " soundcard.");
  module_param_array(enable, bool, NULL, 0444);
  MODULE_PARM_DESC(enable, "Enable " CARD_NAME " soundcard.");
 ]]>
        </programlisting>
      </informalexample>
    </para>
    <para>
      Also, don't forget to define the module description, classes,
      license and devices. Especially, the recent modprobe requires to
      define the module license as GPL, etc., otherwise the system is
      shown as <quote>tainted</quote>. 
      <informalexample>
        <programlisting>
 <![CDATA[
  MODULE_DESCRIPTION("My Chip");
  MODULE_LICENSE("GPL");
  MODULE_SUPPORTED_DEVICE("{{Vendor,My Chip Name}}");
 ]]>
        </programlisting>
      </informalexample>
    </para>
  </chapter>
 <!-- ****************************************************** -->
 <!-- How To Put Your Driver  -->
 <!-- ****************************************************** -->
  <chapter id="how-to-put-your-driver">
    <title>How To Put Your Driver Into ALSA Tree</title>
 	<section>
 	<title>General</title>
 	<para>
 	So far, you've learned how to write the driver codes.
 	And you might have a question now: how to put my own
 	driver into the ALSA driver tree?
 	Here (finally :) the standard procedure is described briefly.
 	</para>
 	<para>
 	Suppose that you'll create a new PCI driver for the card
 	<quote>xyz</quote>.  The card module name would be
 	snd-xyz.  The new driver is usually put into alsa-driver
 	tree, <filename>alsa-driver/pci</filename> directory in
 	the case of PCI cards.
 	Then the driver is evaluated, audited and tested
 	by developers and users.  After a certain time, the driver
 	will go to alsa-kernel tree (to the corresponding directory,
 	such as <filename>alsa-kernel/pci</filename>) and eventually
 	integrated into Linux 2.6 tree (the directory would be
 	<filename>linux/sound/pci</filename>).
 	</para>
 	<para>
 	In the following sections, the driver code is supposed
 	to be put into alsa-driver tree.  The two cases are assumed:
 	a driver consisting of a single source file and one consisting
 	of several source files.
 	</para>
 	</section>
 	<section>
 	<title>Driver with A Single Source File</title>
 	<para>
 	<orderedlist>
 	<listitem>
 	<para>
 	Modify alsa-driver/pci/Makefile
 	</para>
 	<para>
 	Suppose you have a file xyz.c.  Add the following
 	two lines
      <informalexample>
        <programlisting>
 <![CDATA[
  snd-xyz-objs := xyz.o
  obj-$(CONFIG_SND_XYZ) += snd-xyz.o
 ]]>
        </programlisting>
      </informalexample>
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Create the Kconfig entry
 	</para>
 	<para>
 	Add the new entry of Kconfig for your xyz driver.
      <informalexample>
        <programlisting>
 <![CDATA[
  config SND_XYZ
          tristate "Foobar XYZ"
          depends on SND
          select SND_PCM
          help
            Say Y here to include support for Foobar XYZ soundcard.
            To compile this driver as a module, choose M here: the module
            will be called snd-xyz.
 ]]>
        </programlisting>
      </informalexample>
 	the line, select SND_PCM, specifies that the driver xyz supports
 	PCM.  In addition to SND_PCM, the following components are
 	supported for select command:
 	SND_RAWMIDI, SND_TIMER, SND_HWDEP, SND_MPU401_UART,
 	SND_OPL3_LIB, SND_OPL4_LIB, SND_VX_LIB, SND_AC97_CODEC.
 	Add the select command for each supported component.
 	</para>
 	<para>
 	Note that some selections imply the lowlevel selections.
 	For example, PCM includes TIMER, MPU401_UART includes RAWMIDI,
 	AC97_CODEC includes PCM, and OPL3_LIB includes HWDEP.
 	You don't need to give the lowlevel selections again.
 	</para>
 	<para>
 	For the details of Kconfig script, refer to the kbuild
 	documentation.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Run cvscompile script to re-generate the configure script and
 	build the whole stuff again.
 	</para>
 	</listitem>
 	</orderedlist>
 	</para>
 	</section>
 	<section>
 	<title>Drivers with Several Source Files</title>
 	<para>
 	Suppose that the driver snd-xyz have several source files.
 	They are located in the new subdirectory,
 	pci/xyz.
 	<orderedlist>
 	<listitem>
 	<para>
 	Add a new directory (<filename>xyz</filename>) in
 	<filename>alsa-driver/pci/Makefile</filename> like below
      <informalexample>
        <programlisting>
 <![CDATA[
  obj-$(CONFIG_SND) += xyz/
 ]]>
        </programlisting>
      </informalexample>
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Under the directory <filename>xyz</filename>, create a Makefile
      <example>
 	<title>Sample Makefile for a driver xyz</title>
        <programlisting>
 <![CDATA[
  ifndef SND_TOPDIR
  SND_TOPDIR=../..
  endif
  include $(SND_TOPDIR)/toplevel.config
  include $(SND_TOPDIR)/Makefile.conf
  snd-xyz-objs := xyz.o abc.o def.o
  obj-$(CONFIG_SND_XYZ) += snd-xyz.o
  include $(SND_TOPDIR)/Rules.make
 ]]>
        </programlisting>
      </example>
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Create the Kconfig entry
 	</para>
 	<para>
 	This procedure is as same as in the last section.
 	</para>
 	</listitem>
 	<listitem>
 	<para>
 	Run cvscompile script to re-generate the configure script and
 	build the whole stuff again.
 	</para>
 	</listitem>
 	</orderedlist>
 	</para>
 	</section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Useful Functions  -->
 <!-- ****************************************************** -->
  <chapter id="useful-functions">
    <title>Useful Functions</title>
    <section id="useful-functions-snd-printk">
      <title><function>snd_printk()</function> and friends</title>
      <para>
        ALSA provides a verbose version of
      <function>printk()</function> function. If a kernel config
      <constant>CONFIG_SND_VERBOSE_PRINTK</constant> is set, this
      function prints the given message together with the file name
      and the line of the caller. The <constant>KERN_XXX</constant>
      prefix is processed as 
      well as the original <function>printk()</function> does, so it's
      recommended to add this prefix, e.g. 
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_printk(KERN_ERR "Oh my, sorry, it's extremely bad!\n");
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        There are also <function>printk()</function>'s for
      debugging. <function>snd_printd()</function> can be used for
      general debugging purposes. If
      <constant>CONFIG_SND_DEBUG</constant> is set, this function is
      compiled, and works just like
      <function>snd_printk()</function>. If the ALSA is compiled
      without the debugging flag, it's ignored. 
      </para>
      <para>
        <function>snd_printdd()</function> is compiled in only when
      <constant>CONFIG_SND_DEBUG_DETECT</constant> is set. Please note
      that <constant>DEBUG_DETECT</constant> is not set as default
      even if you configure the alsa-driver with
      <option>--with-debug=full</option> option. You need to give
      explicitly <option>--with-debug=detect</option> option instead. 
      </para>
    </section>
    <section id="useful-functions-snd-assert">
      <title><function>snd_assert()</function></title>
      <para>
        <function>snd_assert()</function> macro is similar with the
      normal <function>assert()</function> macro. For example,  
        <informalexample>
          <programlisting>
 <![CDATA[
  snd_assert(pointer != NULL, return -EINVAL);
 ]]>
          </programlisting>
        </informalexample>
      </para>
      <para>
        The first argument is the expression to evaluate, and the
      second argument is the action if it fails. When
      <constant>CONFIG_SND_DEBUG</constant>, is set, it will show an
      error message such as <computeroutput>BUG? (xxx)</computeroutput>
      together with stack trace.
      </para>
      <para>
 	 When no debug flag is set, this macro is ignored. 
      </para>
    </section>
    <section id="useful-functions-snd-bug">
      <title><function>snd_BUG()</function></title>
      <para>
        It shows <computeroutput>BUG?</computeroutput> message and
      stack trace as well as <function>snd_assert</function> at the point.
      It's useful to show that a fatal error happens there. 
      </para>
      <para>
 	 When no debug flag is set, this macro is ignored. 
      </para>
    </section>
  </chapter>
 <!-- ****************************************************** -->
 <!-- Acknowledgments  -->
 <!-- ****************************************************** -->
  <chapter id="acknowledments">
    <title>Acknowledgments</title>
    <para>
      I would like to thank Phil Kerr for his help for improvement and
      corrections of this document. 
    </para>
    <para>
    Kevin Conder reformatted the original plain-text to the
    DocBook format.
    </para>
    <para>
    Giuliano Pochini corrected typos and contributed the example codes
    in the hardware constraints section.
    </para>
  </chapter>
 </book>
 	Installing and using Creative AWE midi sound under Linux.
 This documentation is devoted to the Creative Sound Blaster AWE32, AWE64 and 
 SB32.
 1) Make sure you have an ORIGINAL Creative SB32, AWE32 or AWE64 card. This
   is important, because the driver works only with real Creative cards.
 2) The first thing you need to do is re-compile your kernel with support for
   your sound card. Run your favourite tool to configure the kernel and when
   you get to the "Sound" menu you should enable support for the following:
   Sound card support,
   OSS sound modules,
   100% Sound Blaster compatibles (SB16/32/64, ESS, Jazz16) support,
   AWE32 synth
   If your card is "Plug and Play" you will also need to enable these two
   options, found under the "Plug and Play configuration" menu:
   Plug and Play support
   ISA Plug and Play support
   Now compile and install the kernel in normal fashion. If you don't know
   how to do this you can find instructions for this in the README file
   located in the root directory of the kernel source.
 3) Before you can start playing midi files you will have to load a sound
   bank file. The utility needed for doing this is called "sfxload", and it
   is one of the utilities found in a package called "awesfx". If this
   package is not available in your distribution you can download the AWE
   snapshot from Creative Labs Open Source website:
   http://www.opensource.creative.com/snapshot.html
   Once you have unpacked the AWE snapshot you will see a "awesfx"
   directory. Follow the instructions in awesfx/docs/INSTALL to install the
   utilities in this package. After doing this, sfxload should be installed
   as:
   /usr/local/bin/sfxload
   To enable AWE general midi synthesis you should also get the sound bank
   file for general midi from:
   http://members.xoom.com/yar/synthgm.sbk.gz
   Copy it to a directory of your choice, and unpack it there.
 4) Edit /etc/modprobe.conf, and insert the following lines at the end of the
   file:
  alias sound-slot-0 sb
  alias sound-service-0-1 awe_wave
  install awe_wave /sbin/modprobe --first-time -i awe_wave && /usr/local/bin/sfxload PATH_TO_SOUND_BANK_FILE
  You will of course have to change "PATH_TO_SOUND_BANK_FILE" to the full
-  path of of the sound bank file. That will enable the Sound Blaster and AWE
+  path of the sound bank file. That will enable the Sound Blaster and AWE
  wave synthesis. To play midi files you should get one of these programs if
  you don't already have them:
  Playmidi:			http://playmidi.openprojects.net
  AWEMidi Player (drvmidi)  	Included in the previously mentioned AWE
  				snapshot.
  You will probably have to pass the "-e" switch to playmidi to have it use
  your midi device. drvmidi should work without switches.
  If something goes wrong please e-mail me. All comments and suggestions are
  welcome.
 		    Yaroslav Rosomakho (alons55@dialup.ptt.ru)
 			    http://www.yar.opennet.ru
 Last Updated: Feb 3 2001
 Recording
 ---------
 Recording does not work on the author's card, but there
 is at least one report of it working on later silicon.
 The chip behaves differently than described in the data sheet,
 likely due to a chip bug. Working around this would require
 the help of ESS (for example by publishing an errata sheet),
-but ESS has not done so so far.
+but ESS has not done so far.
 Also, the chip only supports 24 bit addresses for recording,
 which means it cannot work on some Alpha mainboards.
 /proc/sound, /dev/sndstat
 -------------------------
 /proc/sound and /dev/sndstat is not supported by the
 driver. To find out whether the driver succeeded loading,
 check the kernel log (dmesg).
 ALaw/uLaw sample formats
 ------------------------
 This driver does not support the ALaw/uLaw sample formats.
 ALaw is the default mode when opening a sound device
 using OSS/Free. The reason for the lack of support is
 that the hardware does not support these formats, and adding
 conversion routines to the kernel would lead to very ugly
 code in the presence of the mmap interface to the driver.
 And since xquake uses mmap, mmap is considered important :-)
 and no sane application uses ALaw/uLaw these days anyway.
 In short, playing a Sun .au file as follows:
 cat my_file.au > /dev/dsp
 does not work. Instead, you may use the play script from
 Chris Bagwell's sox-12.14 package (or later, available from the URL
 below) to play many different audio file formats.
 The script automatically determines the audio format
 and does do audio conversions if necessary.
 http://home.sprynet.com/sprynet/cbagwell/projects.html
 Blocking vs. nonblocking IO
 ---------------------------
 Unlike OSS/Free this driver honours the O_NONBLOCK file flag
 not only during open, but also during read and write.
 This is an effort to make the sound driver interface more
 regular. Timidity has problems with this; a patch
 is available from http://www.ife.ee.ethz.ch/~sailer/linux/pciaudio.html.
 (Timidity patched will also run on OSS/Free).
 MIDI UART
 ---------
 The driver supports a simple MIDI UART interface, with
 no ioctl's supported.
 MIDI synthesizer
 ----------------
 The card has an OPL compatible FM synthesizer.
 Thomas Sailer
 t.sailer@alumni.ethz.ch
 modprobe sound
 insmod ad1848
 insmod gus io=* irq=* dma=* ...
 This loads the driver for the Gravis Ultrasound family of sound cards.
 The gus module takes the following arguments
 io		I/O address of the Ultrasound card (eg. io=0x220)
 irq		IRQ of the Sound Blaster card 
 dma     	DMA channel for the Sound Blaster
 dma16   	2nd DMA channel, only needed for full duplex operation
 type		1 for PnP card
 gus16		1 for using 16 bit sampling daughter board
 no_wave_dma	Set to disable DMA usage for wavetable (see note)
 db16		???
 no_wave_dma option
 This option defaults to a value of 0, which allows the Ultrasound wavetable
-DSP to use DMA for for playback and downloading samples. This is the same
+DSP to use DMA for playback and downloading samples. This is the same
 as the old behaviour. If set to 1, no DMA is needed for downloading samples,
 and allows owners of a GUS MAX to make use of simultaneous digital audio
 (/dev/dsp), MIDI, and wavetable playback.
 If you have problems in recording with GUS MAX, you could try to use
 just one 8 bit DMA channel. Recording will not work with one DMA
 channel if it's a 16 bit one.
 vwsnd - Sound driver for the Silicon Graphics 320 and 540 Visual
 Workstations' onboard audio.
 Copyright 1999 Silicon Graphics, Inc.  All rights reserved.
 At the time of this writing, March 1999, there are two models of
 Visual Workstation, the 320 and the 540.  This document only describes
 those models.  Future Visual Workstation models may have different
 sound capabilities, and this driver will probably not work on those
 boxes.
 The Visual Workstation has an Analog Devices AD1843 "SoundComm" audio
 codec chip.  The AD1843 is accessed through the Cobalt I/O ASIC, also
-known as Lithium.  This driver programs both both chips.
+known as Lithium.  This driver programs both chips.
 ==============================================================================
 QUICK CONFIGURATION
 	# insmod soundcore
 	# insmod vwsnd
 ==============================================================================
 I/O CONNECTIONS
 On the Visual Workstation, only three of the AD1843 inputs are hooked
 up.  The analog line in jacks are connected to the AD1843's AUX1
 input.  The CD audio lines are connected to the AD1843's AUX2 input.
 The microphone jack is connected to the AD1843's MIC input.  The mic
 jack is mono, but the signal is delivered to both the left and right
 MIC inputs.  You can record in stereo from the mic input, but you will
 get the same signal on both channels (within the limits of A/D
 accuracy).  Full scale on the Line input is +/- 2.0 V.  Full scale on
 the MIC input is 20 dB less, or +/- 0.2 V.
 The AD1843's LOUT1 outputs are connected to the Line Out jacks.  The
 AD1843's HPOUT outputs are connected to the speaker/headphone jack.
 LOUT2 is not connected.  Line out's maximum level is +/- 2.0 V peak to
 peak.  The speaker/headphone out's maximum is +/- 4.0 V peak to peak.
 The AD1843's PCM input channel and one of its output channels (DAC1)
 are connected to Lithium.  The other output channel (DAC2) is not
 connected.
 ==============================================================================
 CAPABILITIES
 The AD1843 has PCM input and output (Pulse Code Modulation, also known
 as wavetable).  PCM input and output can be mono or stereo in any of
 four formats.  The formats are 16 bit signed and 8 bit unsigned,
 u-Law, and A-Law format.  Any sample rate from 4 KHz to 49 KHz is
 available, in 1 Hz increments.
 The AD1843 includes an analog mixer that can mix all three input
 signals (line, mic and CD) into the analog outputs.  The mixer has a
 separate gain control and mute switch for each input.
 There are two outputs, line out and speaker/headphone out.  They
 always produce the same signal, and the speaker always has 3 dB more
 gain than the line out.  The speaker/headphone output can be muted,
 but this driver does not export that function.
 The hardware can sync audio to the video clock, but this driver does
 not have a way to specify syncing to video.
 ==============================================================================
 PROGRAMMING
 This section explains the API supported by the driver.  Also see the
 Open Sound Programming Guide at http://www.opensound.com/pguide/ .
 This section assumes familiarity with that document.
 The driver has two interfaces, an I/O interface and a mixer interface.
 There is no MIDI or sequencer capability.
 ==============================================================================
 PROGRAMMING PCM I/O
 The I/O interface is usually accessed as /dev/audio or /dev/dsp.
 Using the standard Open Sound System (OSS) ioctl calls, the sample
 rate, number of channels, and sample format may be set within the
 limitations described above.  The driver supports triggering.  It also
 supports getting the input and output pointers with one-sample
 accuracy.
 The SNDCTL_DSP_GETCAP ioctl returns these capabilities.
 	DSP_CAP_DUPLEX - driver supports full duplex.
 	DSP_CAP_TRIGGER - driver supports triggering.
 	DSP_CAP_REALTIME - values returned by SNDCTL_DSP_GETIPTR
 	and SNDCTL_DSP_GETOPTR are accurate to a few samples.
 Memory mapping (mmap) is not implemented.
 The driver permits subdivided fragment sizes from 64 to 4096 bytes.
 The number of fragments can be anything from 3 fragments to however
 many fragments fit into 124 kilobytes.  It is up to the user to
 determine how few/small fragments can be used without introducing
 glitches with a given workload.  Linux is not realtime, so we can't
 promise anything.  (sigh...)
 When this driver is switched into or out of mu-Law or A-Law mode on
 output, it may produce an audible click.  This is unavoidable.  To
 prevent clicking, use signed 16-bit mode instead, and convert from
 mu-Law or A-Law format in software.
 ==============================================================================
 PROGRAMMING THE MIXER INTERFACE
 The mixer interface is usually accessed as /dev/mixer.  It is accessed
 through ioctls.  The mixer allows the application to control gain or
 mute several audio signal paths, and also allows selection of the
 recording source.
 Each of the constants described here can be read using the
 MIXER_READ(SOUND_MIXER_xxx) ioctl.  Those that are not read-only can
 also be written using the MIXER_WRITE(SOUND_MIXER_xxx) ioctl.  In most
 cases, <sys/soundcard.h> defines constants SOUND_MIXER_READ_xxx and
 SOUND_MIXER_WRITE_xxx which work just as well.
 SOUND_MIXER_CAPS	Read-only
 This is a mask of optional driver capabilities that are implemented.
 This driver's only capability is SOUND_CAP_EXCL_INPUT, which means
 that only one recording source can be active at a time.
 SOUND_MIXER_DEVMASK	Read-only
 This is a mask of the sound channels.  This driver's channels are PCM,
 LINE, MIC, CD, and RECLEV.
 SOUND_MIXER_STEREODEVS	Read-only
 This is a mask of which sound channels are capable of stereo.  All
 channels are capable of stereo.  (But see caveat on MIC input in I/O
 CONNECTIONS section above).
 SOUND_MIXER_OUTMASK	Read-only
 This is a mask of channels that route inputs through to outputs.
 Those are LINE, MIC, and CD.
 SOUND_MIXER_RECMASK	Read-only
 This is a mask of channels that can be recording sources.  Those are
 PCM, LINE, MIC, CD.
 SOUND_MIXER_PCM		Default: 0x5757 (0 dB)
 This is the gain control for PCM output.  The left and right channel
 gain are controlled independently.  This gain control has 64 levels,
 which range from -82.5 dB to +12.0 dB in 1.5 dB steps.  Those 64
 levels are mapped onto 100 levels at the ioctl, see below.
 SOUND_MIXER_LINE	Default: 0x4a4a (0 dB)
 This is the gain control for mixing the Line In source into the
 outputs.  The left and right channel gain are controlled
 independently.  This gain control has 32 levels, which range from
 -34.5 dB to +12.0 dB in 1.5 dB steps.  Those 32 levels are mapped onto
 100 levels at the ioctl, see below.
 SOUND_MIXER_MIC		Default: 0x4a4a (0 dB)
 This is the gain control for mixing the MIC source into the outputs.
 The left and right channel gain are controlled independently.  This
 gain control has 32 levels, which range from -34.5 dB to +12.0 dB in
 1.5 dB steps.  Those 32 levels are mapped onto 100 levels at the
 ioctl, see below.
 SOUND_MIXER_CD		Default: 0x4a4a (0 dB)
 This is the gain control for mixing the CD audio source into the
 outputs.  The left and right channel gain are controlled
 independently.  This gain control has 32 levels, which range from
 -34.5 dB to +12.0 dB in 1.5 dB steps.  Those 32 levels are mapped onto
 100 levels at the ioctl, see below.
 SOUND_MIXER_RECLEV	 Default: 0 (0 dB)
 This is the gain control for PCM input (RECording LEVel).  The left
 and right channel gain are controlled independently.  This gain
 control has 16 levels, which range from 0 dB to +22.5 dB in 1.5 dB
 steps.  Those 16 levels are mapped onto 100 levels at the ioctl, see
 below.
 SOUND_MIXER_RECSRC	 Default: SOUND_MASK_LINE
 This is a mask of currently selected PCM input sources (RECording
 SouRCes).  Because the AD1843 can only have a single recording source
 at a time, only one bit at a time can be set in this mask.  The
 allowable values are SOUND_MASK_PCM, SOUND_MASK_LINE, SOUND_MASK_MIC,
 or SOUND_MASK_CD.  Selecting SOUND_MASK_PCM sets up internal
 resampling which is useful for loopback testing and for hardware
 sample rate conversion.  But software sample rate conversion is
 probably faster, so I don't know how useful that is.
 SOUND_MIXER_OUTSRC	DEFAULT: SOUND_MASK_LINE|SOUND_MASK_MIC|SOUND_MASK_CD
 This is a mask of sources that are currently passed through to the
 outputs.  Those sources whose bits are not set are muted.
 ==============================================================================
 GAIN CONTROL
 There are five gain controls listed above.  Each has 16, 32, or 64
 steps.  Each control has 1.5 dB of gain per step.  Each control is
 stereo.
 The OSS defines the argument to a channel gain ioctl as having two
 components, left and right, each of which ranges from 0 to 100.  The
 two components are packed into the same word, with the left side gain
 in the least significant byte, and the right side gain in the second
 least significant byte.  In C, we would say this.
 	#include <assert.h>
 	...
 	 	assert(leftgain >= 0 && leftgain <= 100);
 		assert(rightgain >= 0 && rightgain <= 100);
 		arg = leftgain | rightgain << 8;
 So each OSS gain control has 101 steps.  But the hardware has 16, 32,
 or 64 steps.  The hardware steps are spread across the 101 OSS steps
 nearly evenly.  The conversion formulas are like this, given N equals
 16, 32, or 64.
 	int round = N/2 - 1;
 	OSS_gain_steps = (hw_gain_steps * 100 + round) / (N - 1);
 	hw_gain_steps = (OSS_gain_steps * (N - 1) + round) / 100;
 Here is a snippet of C code that will return the left and right gain
 of any channel in dB.  Pass it one of the predefined gain_desc_t
 structures to access any of the five channels' gains.
 	typedef struct gain_desc {
 		float min_gain;
 		float gain_step;
 		int nbits;
 		int chan;
 	} gain_desc_t;
 	const gain_desc_t gain_pcm    = { -82.5, 1.5, 6, SOUND_MIXER_PCM    };
 	const gain_desc_t gain_line   = { -34.5, 1.5, 5, SOUND_MIXER_LINE   };
 	const gain_desc_t gain_mic    = { -34.5, 1.5, 5, SOUND_MIXER_MIC    };
 	const gain_desc_t gain_cd     = { -34.5, 1.5, 5, SOUND_MIXER_CD     };
 	const gain_desc_t gain_reclev = {   0.0, 1.5, 4, SOUND_MIXER_RECLEV };
 	int get_gain_dB(int fd, const gain_desc_t *gp,
 			float *left, float *right)
 	{
 		int word;
 		int lg, rg;
 		int mask = (1 << gp->nbits) - 1;
 		if (ioctl(fd, MIXER_READ(gp->chan), &word) != 0)
 			return -1;	/* fail */
 		lg = word & 0xFF;
 		rg = word >> 8 & 0xFF;
 		lg = (lg * mask + mask / 2) / 100;
 		rg = (rg * mask + mask / 2) / 100;
 		*left = gp->min_gain + gp->gain_step * lg;
 		*right = gp->min_gain + gp->gain_step * rg;
 		return 0;
 	}	
 And here is the corresponding routine to set a channel's gain in dB.
 	int set_gain_dB(int fd, const gain_desc_t *gp, float left, float right)
 	{
 		float max_gain =
 			gp->min_gain + (1 << gp->nbits) * gp->gain_step;
 		float round = gp->gain_step / 2;
 		int mask = (1 << gp->nbits) - 1;
 		int word;
 		int lg, rg;
 		if (left < gp->min_gain || right < gp->min_gain)
 			return EINVAL;
 		lg = (left - gp->min_gain + round) / gp->gain_step;
 		rg = (right - gp->min_gain + round) / gp->gain_step;
 		if (lg >= (1 << gp->nbits) || rg >= (1 << gp->nbits))
 			return EINVAL;
 		lg = (100 * lg + mask / 2) / mask;
 		rg = (100 * rg + mask / 2) / mask;
 		word = lg | rg << 8;
 		return ioctl(fd, MIXER_WRITE(gp->chan), &word);
 	}
 PXA2xx SPI on SSP driver HOWTO
 ===================================================
 This a mini howto on the pxa2xx_spi driver.  The driver turns a PXA2xx
 synchronous serial port into a SPI master controller
 (see Documentation/spi/spi_summary). The driver has the following features
 - Support for any PXA2xx SSP
 - SSP PIO and SSP DMA data transfers.
 - External and Internal (SSPFRM) chip selects.
 - Per slave device (chip) configuration.
 - Full suspend, freeze, resume support.
 The driver is built around a "spi_message" fifo serviced by workqueue and a
 tasklet. The workqueue, "pump_messages", drives message fifo and the tasklet
 (pump_transfer) is responsible for queuing SPI transactions and setting up and
 launching the dma/interrupt driven transfers.
 Declaring PXA2xx Master Controllers
 -----------------------------------
 Typically a SPI master is defined in the arch/.../mach-*/board-*.c as a
 "platform device".  The master configuration is passed to the driver via a table
 found in include/asm-arm/arch-pxa/pxa2xx_spi.h:
 struct pxa2xx_spi_master {
 	enum pxa_ssp_type ssp_type;
 	u32 clock_enable;
 	u16 num_chipselect;
 	u8 enable_dma;
 };
 The "pxa2xx_spi_master.ssp_type" field must have a value between 1 and 3 and
 informs the driver which features a particular SSP supports.
 The "pxa2xx_spi_master.clock_enable" field is used to enable/disable the
 corresponding SSP peripheral block in the "Clock Enable Register (CKEN"). See
 the "PXA2xx Developer Manual" section "Clocks and Power Management".
 The "pxa2xx_spi_master.num_chipselect" field is used to determine the number of
 slave device (chips) attached to this SPI master.
 The "pxa2xx_spi_master.enable_dma" field informs the driver that SSP DMA should
 be used.  This caused the driver to acquire two DMA channels: rx_channel and
 tx_channel.  The rx_channel has a higher DMA service priority the tx_channel.
 See the "PXA2xx Developer Manual" section "DMA Controller".
 NSSP MASTER SAMPLE
 ------------------
 Below is a sample configuration using the PXA255 NSSP.
 static struct resource pxa_spi_nssp_resources[] = {
 	[0] = {
 		.start	= __PREG(SSCR0_P(2)), /* Start address of NSSP */
 		.end	= __PREG(SSCR0_P(2)) + 0x2c, /* Range of registers */
 		.flags	= IORESOURCE_MEM,
 	},
 	[1] = {
 		.start	= IRQ_NSSP, /* NSSP IRQ */
 		.end	= IRQ_NSSP,
 		.flags	= IORESOURCE_IRQ,
 	},
 };
 static struct pxa2xx_spi_master pxa_nssp_master_info = {
 	.ssp_type = PXA25x_NSSP, /* Type of SSP */
 	.clock_enable = CKEN9_NSSP, /* NSSP Peripheral clock */
 	.num_chipselect = 1, /* Matches the number of chips attached to NSSP */
 	.enable_dma = 1, /* Enables NSSP DMA */
 };
 static struct platform_device pxa_spi_nssp = {
 	.name = "pxa2xx-spi", /* MUST BE THIS VALUE, so device match driver */
 	.id = 2, /* Bus number, MUST MATCH SSP number 1..n */
 	.resource = pxa_spi_nssp_resources,
 	.num_resources = ARRAY_SIZE(pxa_spi_nssp_resources),
 	.dev = {
 		.platform_data = &pxa_nssp_master_info, /* Passed to driver */
 	},
 };
 static struct platform_device *devices[] __initdata = {
 	&pxa_spi_nssp,
 };
 static void __init board_init(void)
 {
 	(void)platform_add_device(devices, ARRAY_SIZE(devices));
 }
 Declaring Slave Devices
 -----------------------
 Typically each SPI slave (chip) is defined in the arch/.../mach-*/board-*.c
 using the "spi_board_info" structure found in "linux/spi/spi.h". See
 "Documentation/spi/spi_summary" for additional information.
 Each slave device attached to the PXA must provide slave specific configuration
 information via the structure "pxa2xx_spi_chip" found in
 "include/asm-arm/arch-pxa/pxa2xx_spi.h".  The pxa2xx_spi master controller driver
 will uses the configuration whenever the driver communicates with the slave
 device.
 struct pxa2xx_spi_chip {
 	u8 tx_threshold;
 	u8 rx_threshold;
 	u8 dma_burst_size;
 	u32 timeout_microsecs;
 	u8 enable_loopback;
 	void (*cs_control)(u32 command);
 };
 The "pxa2xx_spi_chip.tx_threshold" and "pxa2xx_spi_chip.rx_threshold" fields are
 used to configure the SSP hardware fifo.  These fields are critical to the
 performance of pxa2xx_spi driver and misconfiguration will result in rx
 fifo overruns (especially in PIO mode transfers). Good default values are
 	.tx_threshold = 12,
 	.rx_threshold = 4,
 The "pxa2xx_spi_chip.dma_burst_size" field is used to configure PXA2xx DMA
 engine and is related the "spi_device.bits_per_word" field.  Read and understand
 the PXA2xx "Developer Manual" sections on the DMA controller and SSP Controllers
 to determine the correct value. An SSP configured for byte-wide transfers would
 use a value of 8.
 The "pxa2xx_spi_chip.timeout_microsecs" fields is used to efficiently handle
 trailing bytes in the SSP receiver fifo.  The correct value for this field is
 dependent on the SPI bus speed ("spi_board_info.max_speed_hz") and the specific
-slave device.  Please note the the PXA2xx SSP 1 does not support trailing byte
+slave device.  Please note that the PXA2xx SSP 1 does not support trailing byte
 timeouts and must busy-wait any trailing bytes.
 The "pxa2xx_spi_chip.enable_loopback" field is used to place the SSP porting
 into internal loopback mode.  In this mode the SSP controller internally
-connects the SSPTX pin the the SSPRX pin.  This is useful for initial setup
+connects the SSPTX pin to the SSPRX pin.  This is useful for initial setup
 testing.
 The "pxa2xx_spi_chip.cs_control" field is used to point to a board specific
 function for asserting/deasserting a slave device chip select.  If the field is
 NULL, the pxa2xx_spi master controller driver assumes that the SSP port is
 configured to use SSPFRM instead.
 NSSP SALVE SAMPLE
 -----------------
 The pxa2xx_spi_chip structure is passed to the pxa2xx_spi driver in the
 "spi_board_info.controller_data" field. Below is a sample configuration using
 the PXA255 NSSP.
 /* Chip Select control for the CS8415A SPI slave device */
 static void cs8415a_cs_control(u32 command)
 {
 	if (command & PXA2XX_CS_ASSERT)
 		GPCR(2) = GPIO_bit(2);
 	else
 		GPSR(2) = GPIO_bit(2);
 }
 /* Chip Select control for the CS8405A SPI slave device */
 static void cs8405a_cs_control(u32 command)
 {
 	if (command & PXA2XX_CS_ASSERT)
 		GPCR(3) = GPIO_bit(3);
 	else
 		GPSR(3) = GPIO_bit(3);
 }
 static struct pxa2xx_spi_chip cs8415a_chip_info = {
 	.tx_threshold = 12, /* SSP hardward FIFO threshold */
 	.rx_threshold = 4, /* SSP hardward FIFO threshold */
 	.dma_burst_size = 8, /* Byte wide transfers used so 8 byte bursts */
 	.timeout_microsecs = 64, /* Wait at least 64usec to handle trailing */
 	.cs_control = cs8415a_cs_control, /* Use external chip select */
 };
 static struct pxa2xx_spi_chip cs8405a_chip_info = {
 	.tx_threshold = 12, /* SSP hardward FIFO threshold */
 	.rx_threshold = 4, /* SSP hardward FIFO threshold */
 	.dma_burst_size = 8, /* Byte wide transfers used so 8 byte bursts */
 	.timeout_microsecs = 64, /* Wait at least 64usec to handle trailing */
 	.cs_control = cs8405a_cs_control, /* Use external chip select */
 };
 static struct spi_board_info streetracer_spi_board_info[] __initdata = {
 	{
 		.modalias = "cs8415a", /* Name of spi_driver for this device */
 		.max_speed_hz = 3686400, /* Run SSP as fast a possbile */
 		.bus_num = 2, /* Framework bus number */
 		.chip_select = 0, /* Framework chip select */
 		.platform_data = NULL; /* No spi_driver specific config */
 		.controller_data = &cs8415a_chip_info, /* Master chip config */
 		.irq = STREETRACER_APCI_IRQ, /* Slave device interrupt */
 	},
 	{
 		.modalias = "cs8405a", /* Name of spi_driver for this device */
 		.max_speed_hz = 3686400, /* Run SSP as fast a possbile */
 		.bus_num = 2, /* Framework bus number */
 		.chip_select = 1, /* Framework chip select */
 		.controller_data = &cs8405a_chip_info, /* Master chip config */
 		.irq = STREETRACER_APCI_IRQ, /* Slave device interrupt */
 	},
 };
 static void __init streetracer_init(void)
 {
 	spi_register_board_info(streetracer_spi_board_info,
 				ARRAY_SIZE(streetracer_spi_board_info));
 }
 DMA and PIO I/O Support
 -----------------------
 The pxa2xx_spi driver support both DMA and interrupt driven PIO message
 transfers.  The driver defaults to PIO mode and DMA transfers must enabled by
-setting the "enable_dma" flag in the "pxa2xx_spi_master" structure and and
+setting the "enable_dma" flag in the "pxa2xx_spi_master" structure and
 ensuring that the "pxa2xx_spi_chip.dma_burst_size" field is non-zero.  The DMA
 mode support both coherent and stream based DMA mappings.
 The following logic is used to determine the type of I/O to be used on
 a per "spi_transfer" basis:
 if !enable_dma or dma_burst_size == 0 then
 	always use PIO transfers
 if spi_message.is_dma_mapped and rx_dma_buf != 0 and tx_dma_buf != 0 then
 	use coherent DMA mode
 if rx_buf and tx_buf are aligned on 8 byte boundary then
 	use streaming DMA mode
 otherwise
 	use PIO transfer
 THANKS TO
 ---------
 David Brownell and others for mentoring the development of this driver.
 Overview of Linux kernel SPI support
 ====================================
 02-Dec-2005
 What is SPI?
 ------------
 The "Serial Peripheral Interface" (SPI) is a synchronous four wire serial
 link used to connect microcontrollers to sensors, memory, and peripherals.
 The three signal wires hold a clock (SCLK, often on the order of 10 MHz),
 and parallel data lines with "Master Out, Slave In" (MOSI) or "Master In,
 Slave Out" (MISO) signals.  (Other names are also used.)  There are four
 clocking modes through which data is exchanged; mode-0 and mode-3 are most
 commonly used.  Each clock cycle shifts data out and data in; the clock
 doesn't cycle except when there is data to shift.
 SPI masters may use a "chip select" line to activate a given SPI slave
 device, so those three signal wires may be connected to several chips
 in parallel.  All SPI slaves support chipselects.  Some devices have
 other signals, often including an interrupt to the master.
 Unlike serial busses like USB or SMBUS, even low level protocols for
 SPI slave functions are usually not interoperable between vendors
 (except for cases like SPI memory chips).
  - SPI may be used for request/response style device protocols, as with
    touchscreen sensors and memory chips.
  - It may also be used to stream data in either direction (half duplex),
    or both of them at the same time (full duplex).
  - Some devices may use eight bit words.  Others may different word
    lengths, such as streams of 12-bit or 20-bit digital samples.
 In the same way, SPI slaves will only rarely support any kind of automatic
 discovery/enumeration protocol.  The tree of slave devices accessible from
 a given SPI master will normally be set up manually, with configuration
 tables.
 SPI is only one of the names used by such four-wire protocols, and
 most controllers have no problem handling "MicroWire" (think of it as
 half-duplex SPI, for request/response protocols), SSP ("Synchronous
 Serial Protocol"), PSP ("Programmable Serial Protocol"), and other
 related protocols.
 Microcontrollers often support both master and slave sides of the SPI
 protocol.  This document (and Linux) currently only supports the master
 side of SPI interactions.
 Who uses it?  On what kinds of systems?
 ---------------------------------------
 Linux developers using SPI are probably writing device drivers for embedded
 systems boards.  SPI is used to control external chips, and it is also a
 protocol supported by every MMC or SD memory card.  (The older "DataFlash"
 cards, predating MMC cards but using the same connectors and card shape,
 support only SPI.)  Some PC hardware uses SPI flash for BIOS code.
 SPI slave chips range from digital/analog converters used for analog
 sensors and codecs, to memory, to peripherals like USB controllers
 or Ethernet adapters; and more.
 Most systems using SPI will integrate a few devices on a mainboard.
 Some provide SPI links on expansion connectors; in cases where no
 dedicated SPI controller exists, GPIO pins can be used to create a
 low speed "bitbanging" adapter.  Very few systems will "hotplug" an SPI
 controller; the reasons to use SPI focus on low cost and simple operation,
 and if dynamic reconfiguration is important, USB will often be a more
 appropriate low-pincount peripheral bus.
 Many microcontrollers that can run Linux integrate one or more I/O
 interfaces with SPI modes.  Given SPI support, they could use MMC or SD
 cards without needing a special purpose MMC/SD/SDIO controller.
 How do these driver programming interfaces work?
 ------------------------------------------------
 The <linux/spi/spi.h> header file includes kerneldoc, as does the
 main source code, and you should certainly read that.  This is just
 an overview, so you get the big picture before the details.
 SPI requests always go into I/O queues.  Requests for a given SPI device
 are always executed in FIFO order, and complete asynchronously through
 completion callbacks.  There are also some simple synchronous wrappers
 for those calls, including ones for common transaction types like writing
 a command and then reading its response.
 There are two types of SPI driver, here called:
  Controller drivers ... these are often built in to System-On-Chip
 	processors, and often support both Master and Slave roles.
 	These drivers touch hardware registers and may use DMA.
 	Or they can be PIO bitbangers, needing just GPIO pins.
  Protocol drivers ... these pass messages through the controller
 	driver to communicate with a Slave or Master device on the
 	other side of an SPI link.
 So for example one protocol driver might talk to the MTD layer to export
 data to filesystems stored on SPI flash like DataFlash; and others might
 control audio interfaces, present touchscreen sensors as input interfaces,
 or monitor temperature and voltage levels during industrial processing.
 And those might all be sharing the same controller driver.
 A "struct spi_device" encapsulates the master-side interface between
 those two types of driver.  At this writing, Linux has no slave side
 programming interface.
 There is a minimal core of SPI programming interfaces, focussing on
 using driver model to connect controller and protocol drivers using
 device tables provided by board specific initialization code.  SPI
 shows up in sysfs in several locations:
   /sys/devices/.../CTLR/spiB.C ... spi_device for on bus "B",
 	chipselect C, accessed through CTLR.
   /sys/devices/.../CTLR/spiB.C/modalias ... identifies the driver
 	that should be used with this device (for hotplug/coldplug)
   /sys/bus/spi/devices/spiB.C ... symlink to the physical
   	spiB-C device
   /sys/bus/spi/drivers/D ... driver for one or more spi*.* devices
   /sys/class/spi_master/spiB ... class device for the controller
 	managing bus "B".  All the spiB.* devices share the same
 	physical SPI bus segment, with SCLK, MOSI, and MISO.
 How does board-specific init code declare SPI devices?
 ------------------------------------------------------
 Linux needs several kinds of information to properly configure SPI devices.
 That information is normally provided by board-specific code, even for
 chips that do support some of automated discovery/enumeration.
 DECLARE CONTROLLERS
 The first kind of information is a list of what SPI controllers exist.
 For System-on-Chip (SOC) based boards, these will usually be platform
 devices, and the controller may need some platform_data in order to
 operate properly.  The "struct platform_device" will include resources
 like the physical address of the controller's first register and its IRQ.
 Platforms will often abstract the "register SPI controller" operation,
 maybe coupling it with code to initialize pin configurations, so that
 the arch/.../mach-*/board-*.c files for several boards can all share the
 same basic controller setup code.  This is because most SOCs have several
 SPI-capable controllers, and only the ones actually usable on a given
 board should normally be set up and registered.
 So for example arch/.../mach-*/board-*.c files might have code like:
 	#include <asm/arch/spi.h>	/* for mysoc_spi_data */
 	/* if your mach-* infrastructure doesn't support kernels that can
 	 * run on multiple boards, pdata wouldn't benefit from "__init".
 	 */
 	static struct mysoc_spi_data __init pdata = { ... };
 	static __init board_init(void)
 	{
 		...
 		/* this board only uses SPI controller #2 */
 		mysoc_register_spi(2, &pdata);
 		...
 	}
 And SOC-specific utility code might look something like:
 	#include <asm/arch/spi.h>
 	static struct platform_device spi2 = { ... };
 	void mysoc_register_spi(unsigned n, struct mysoc_spi_data *pdata)
 	{
 		struct mysoc_spi_data *pdata2;
 		pdata2 = kmalloc(sizeof *pdata2, GFP_KERNEL);
 		*pdata2 = pdata;
 		...
 		if (n == 2) {
 			spi2->dev.platform_data = pdata2;
 			register_platform_device(&spi2);
 			/* also: set up pin modes so the spi2 signals are
 			 * visible on the relevant pins ... bootloaders on
 			 * production boards may already have done this, but
 			 * developer boards will often need Linux to do it.
 			 */
 		}
 		...
 	}
 Notice how the platform_data for boards may be different, even if the
 same SOC controller is used.  For example, on one board SPI might use
 an external clock, where another derives the SPI clock from current
 settings of some master clock.
 DECLARE SLAVE DEVICES
 The second kind of information is a list of what SPI slave devices exist
 on the target board, often with some board-specific data needed for the
 driver to work correctly.
 Normally your arch/.../mach-*/board-*.c files would provide a small table
 listing the SPI devices on each board.  (This would typically be only a
 small handful.)  That might look like:
 	static struct ads7846_platform_data ads_info = {
 		.vref_delay_usecs	= 100,
 		.x_plate_ohms		= 580,
 		.y_plate_ohms		= 410,
 	};
 	static struct spi_board_info spi_board_info[] __initdata = {
 	{
 		.modalias	= "ads7846",
 		.platform_data	= &ads_info,
 		.mode		= SPI_MODE_0,
 		.irq		= GPIO_IRQ(31),
 		.max_speed_hz	= 120000 /* max sample rate at 3V */ * 16,
 		.bus_num	= 1,
 		.chip_select	= 0,
 	},
 	};
 Again, notice how board-specific information is provided; each chip may need
 several types.  This example shows generic constraints like the fastest SPI
 clock to allow (a function of board voltage in this case) or how an IRQ pin
 is wired, plus chip-specific constraints like an important delay that's
 changed by the capacitance at one pin.
 (There's also "controller_data", information that may be useful to the
 controller driver.  An example would be peripheral-specific DMA tuning
 data or chipselect callbacks.  This is stored in spi_device later.)
 The board_info should provide enough information to let the system work
 without the chip's driver being loaded.  The most troublesome aspect of
 that is likely the SPI_CS_HIGH bit in the spi_device.mode field, since
 sharing a bus with a device that interprets chipselect "backwards" is
 not possible.
 Then your board initialization code would register that table with the SPI
 infrastructure, so that it's available later when the SPI master controller
 driver is registered:
 	spi_register_board_info(spi_board_info, ARRAY_SIZE(spi_board_info));
 Like with other static board-specific setup, you won't unregister those.
 The widely used "card" style computers bundle memory, cpu, and little else
 onto a card that's maybe just thirty square centimeters.  On such systems,
 your arch/.../mach-.../board-*.c file would primarily provide information
 about the devices on the mainboard into which such a card is plugged.  That
 certainly includes SPI devices hooked up through the card connectors!
 NON-STATIC CONFIGURATIONS
 Developer boards often play by different rules than product boards, and one
 example is the potential need to hotplug SPI devices and/or controllers.
-For those cases you might need to use use spi_busnum_to_master() to look
+For those cases you might need to use spi_busnum_to_master() to look
 up the spi bus master, and will likely need spi_new_device() to provide the
 board info based on the board that was hotplugged.  Of course, you'd later
 call at least spi_unregister_device() when that board is removed.
 When Linux includes support for MMC/SD/SDIO/DataFlash cards through SPI, those
 configurations will also be dynamic.  Fortunately, those devices all support
 basic device identification probes, so that support should hotplug normally.
 How do I write an "SPI Protocol Driver"?
 ----------------------------------------
 All SPI drivers are currently kernel drivers.  A userspace driver API
 would just be another kernel driver, probably offering some lowlevel
 access through aio_read(), aio_write(), and ioctl() calls and using the
 standard userspace sysfs mechanisms to bind to a given SPI device.
 SPI protocol drivers somewhat resemble platform device drivers:
 	static struct spi_driver CHIP_driver = {
 		.driver = {
 			.name		= "CHIP",
 			.bus		= &spi_bus_type,
 			.owner		= THIS_MODULE,
 		},
 		.probe		= CHIP_probe,
 		.remove		= __devexit_p(CHIP_remove),
 		.suspend	= CHIP_suspend,
 		.resume		= CHIP_resume,
 	};
 The driver core will autmatically attempt to bind this driver to any SPI
 device whose board_info gave a modalias of "CHIP".  Your probe() code
 might look like this unless you're creating a class_device:
 	static int __devinit CHIP_probe(struct spi_device *spi)
 	{
 		struct CHIP			*chip;
 		struct CHIP_platform_data	*pdata;
 		/* assuming the driver requires board-specific data: */
 		pdata = &spi->dev.platform_data;
 		if (!pdata)
 			return -ENODEV;
 		/* get memory for driver's per-chip state */
 		chip = kzalloc(sizeof *chip, GFP_KERNEL);
 		if (!chip)
 			return -ENOMEM;
 		dev_set_drvdata(&spi->dev, chip);
 		... etc
 		return 0;
 	}
 As soon as it enters probe(), the driver may issue I/O requests to
 the SPI device using "struct spi_message".  When remove() returns,
 the driver guarantees that it won't submit any more such messages.
-  - An spi_message is a sequence of of protocol operations, executed
+  - An spi_message is a sequence of protocol operations, executed
    as one atomic sequence.  SPI driver controls include:
      + when bidirectional reads and writes start ... by how its
        sequence of spi_transfer requests is arranged;
      + optionally defining short delays after transfers ... using
        the spi_transfer.delay_usecs setting;
      + whether the chipselect becomes inactive after a transfer and
        any delay ... by using the spi_transfer.cs_change flag;
      + hinting whether the next message is likely to go to this same
        device ... using the spi_transfer.cs_change flag on the last
 	transfer in that atomic group, and potentially saving costs
 	for chip deselect and select operations.
  - Follow standard kernel rules, and provide DMA-safe buffers in
    your messages.  That way controller drivers using DMA aren't forced
    to make extra copies unless the hardware requires it (e.g. working
    around hardware errata that force the use of bounce buffering).
    If standard dma_map_single() handling of these buffers is inappropriate,
    you can use spi_message.is_dma_mapped to tell the controller driver
    that you've already provided the relevant DMA addresses.
  - The basic I/O primitive is spi_async().  Async requests may be
    issued in any context (irq handler, task, etc) and completion
    is reported using a callback provided with the message.
    After any detected error, the chip is deselected and processing
    of that spi_message is aborted.
  - There are also synchronous wrappers like spi_sync(), and wrappers
    like spi_read(), spi_write(), and spi_write_then_read().  These
    may be issued only in contexts that may sleep, and they're all
    clean (and small, and "optional") layers over spi_async().
  - The spi_write_then_read() call, and convenience wrappers around
    it, should only be used with small amounts of data where the
    cost of an extra copy may be ignored.  It's designed to support
    common RPC-style requests, such as writing an eight bit command
    and reading a sixteen bit response -- spi_w8r16() being one its
    wrappers, doing exactly that.
 Some drivers may need to modify spi_device characteristics like the
 transfer mode, wordsize, or clock rate.  This is done with spi_setup(),
 which would normally be called from probe() before the first I/O is
 done to the device.
 While "spi_device" would be the bottom boundary of the driver, the
 upper boundaries might include sysfs (especially for sensor readings),
 the input layer, ALSA, networking, MTD, the character device framework,
 or other Linux subsystems.
 Note that there are two types of memory your driver must manage as part
 of interacting with SPI devices.
  - I/O buffers use the usual Linux rules, and must be DMA-safe.
    You'd normally allocate them from the heap or free page pool.
    Don't use the stack, or anything that's declared "static".
  - The spi_message and spi_transfer metadata used to glue those
    I/O buffers into a group of protocol transactions.  These can
    be allocated anywhere it's convenient, including as part of
    other allocate-once driver data structures.  Zero-init these.
 If you like, spi_message_alloc() and spi_message_free() convenience
 routines are available to allocate and zero-initialize an spi_message
 with several transfers.
 How do I write an "SPI Master Controller Driver"?
 -------------------------------------------------
 An SPI controller will probably be registered on the platform_bus; write
 a driver to bind to the device, whichever bus is involved.
 The main task of this type of driver is to provide an "spi_master".
 Use spi_alloc_master() to allocate the master, and class_get_devdata()
 to get the driver-private data allocated for that device.
 	struct spi_master	*master;
 	struct CONTROLLER	*c;
 	master = spi_alloc_master(dev, sizeof *c);
 	if (!master)
 		return -ENODEV;
 	c = class_get_devdata(&master->cdev);
 The driver will initialize the fields of that spi_master, including the
 bus number (maybe the same as the platform device ID) and three methods
 used to interact with the SPI core and SPI protocol drivers.  It will
 also initialize its own internal state.  (See below about bus numbering
 and those methods.)
 After you initialize the spi_master, then use spi_register_master() to
 publish it to the rest of the system.  At that time, device nodes for
 the controller and any predeclared spi devices will be made available,
 and the driver model core will take care of binding them to drivers.
 If you need to remove your SPI controller driver, spi_unregister_master()
 will reverse the effect of spi_register_master().
 BUS NUMBERING
 Bus numbering is important, since that's how Linux identifies a given
 SPI bus (shared SCK, MOSI, MISO).  Valid bus numbers start at zero.  On
 SOC systems, the bus numbers should match the numbers defined by the chip
 manufacturer.  For example, hardware controller SPI2 would be bus number 2,
 and spi_board_info for devices connected to it would use that number.
 If you don't have such hardware-assigned bus number, and for some reason
 you can't just assign them, then provide a negative bus number.  That will
 then be replaced by a dynamically assigned number. You'd then need to treat
 this as a non-static configuration (see above).
 SPI MASTER METHODS
    master->setup(struct spi_device *spi)
 	This sets up the device clock rate, SPI mode, and word sizes.
 	Drivers may change the defaults provided by board_info, and then
 	call spi_setup(spi) to invoke this routine.  It may sleep.
    master->transfer(struct spi_device *spi, struct spi_message *message)
    	This must not sleep.  Its responsibility is arrange that the
 	transfer happens and its complete() callback is issued; the two
 	will normally happen later, after other transfers complete.
    master->cleanup(struct spi_device *spi)
 	Your controller driver may use spi_device.controller_state to hold
 	state it dynamically associates with that device.  If you do that,
 	be sure to provide the cleanup() method to free that state.
 SPI MESSAGE QUEUE
 The bulk of the driver will be managing the I/O queue fed by transfer().
 That queue could be purely conceptual.  For example, a driver used only
 for low-frequency sensor acess might be fine using synchronous PIO.
 But the queue will probably be very real, using message->queue, PIO,
 often DMA (especially if the root filesystem is in SPI flash), and
 execution contexts like IRQ handlers, tasklets, or workqueues (such
 as keventd).  Your driver can be as fancy, or as simple, as you need.
 Such a transfer() method would normally just add the message to a
 queue, and then start some asynchronous transfer engine (unless it's
 already running).
 THANKS TO
 ---------
 Contributors to Linux-SPI discussions include (in alphabetical order,
 by last name):
 David Brownell
 Russell King
 Dmitry Pervushin
 Stephen Street
 Mark Underwood
 Andrew Victor
 Vitaly Wool
 unshare system call:
 --------------------
 This document describes the new system call, unshare. The document
 provides an overview of the feature, why it is needed, how it can
 be used, its interface specification, design, implementation and
 how it can be tested.
 Change Log:
 -----------
 version 0.1  Initial document, Janak Desai (janak@us.ibm.com), Jan 11, 2006
 Contents:
 ---------
 	1) Overview
 	2) Benefits
 	3) Cost
 	4) Requirements
 	5) Functional Specification
 	6) High Level Design
 	7) Low Level Design
 	8) Test Specification
 	9) Future Work
 1) Overview
 -----------
 Most legacy operating system kernels support an abstraction of threads
 as multiple execution contexts within a process. These kernels provide
 special resources and mechanisms to maintain these "threads". The Linux
 kernel, in a clever and simple manner, does not make distinction
 between processes and "threads". The kernel allows processes to share
 resources and thus they can achieve legacy "threads" behavior without
 requiring additional data structures and mechanisms in the kernel. The
 power of implementing threads in this manner comes not only from
 its simplicity but also from allowing application programmers to work
 outside the confinement of all-or-nothing shared resources of legacy
 threads. On Linux, at the time of thread creation using the clone system
 call, applications can selectively choose which resources to share
 between threads.
 unshare system call adds a primitive to the Linux thread model that
 allows threads to selectively 'unshare' any resources that were being
 shared at the time of their creation. unshare was conceptualized by
 Al Viro in the August of 2000, on the Linux-Kernel mailing list, as part
 of the discussion on POSIX threads on Linux.  unshare augments the
 usefulness of Linux threads for applications that would like to control
 shared resources without creating a new process. unshare is a natural
 addition to the set of available primitives on Linux that implement
 the concept of process/thread as a virtual machine.
 2) Benefits
 -----------
 unshare would be useful to large application frameworks such as PAM
 where creating a new process to control sharing/unsharing of process
 resources is not possible. Since namespaces are shared by default
 when creating a new process using fork or clone, unshare can benefit
 even non-threaded applications if they have a need to disassociate
 from default shared namespace. The following lists two use-cases
 where unshare can be used.
 2.1 Per-security context namespaces
 -----------------------------------
 unshare can be used to implement polyinstantiated directories using
 the kernel's per-process namespace mechanism. Polyinstantiated directories,
 such as per-user and/or per-security context instance of /tmp, /var/tmp or
 per-security context instance of a user's home directory, isolate user
 processes when working with these directories. Using unshare, a PAM
 module can easily setup a private namespace for a user at login.
 Polyinstantiated directories are required for Common Criteria certification
 with Labeled System Protection Profile, however, with the availability
 of shared-tree feature in the Linux kernel, even regular Linux systems
 can benefit from setting up private namespaces at login and
 polyinstantiating /tmp, /var/tmp and other directories deemed
 appropriate by system administrators.
 2.2 unsharing of virtual memory and/or open files
 -------------------------------------------------
 Consider a client/server application where the server is processing
 client requests by creating processes that share resources such as
 virtual memory and open files. Without unshare, the server has to
 decide what needs to be shared at the time of creating the process
 which services the request. unshare allows the server an ability to
 disassociate parts of the context during the servicing of the
 request. For large and complex middleware application frameworks, this
 ability to unshare after the process was created can be very
 useful.
 3) Cost
 -------
 In order to not duplicate code and to handle the fact that unshare
 works on an active task (as opposed to clone/fork working on a newly
 allocated inactive task) unshare had to make minor reorganizational
 changes to copy_* functions utilized by clone/fork system call.
 There is a cost associated with altering existing, well tested and
 stable code to implement a new feature that may not get exercised
 extensively in the beginning. However, with proper design and code
 review of the changes and creation of an unshare test for the LTP
 the benefits of this new feature can exceed its cost.
 4) Requirements
 ---------------
 unshare reverses sharing that was done using clone(2) system call,
 so unshare should have a similar interface as clone(2). That is,
 since flags in clone(int flags, void *stack) specifies what should
 be shared, similar flags in unshare(int flags) should specify
 what should be unshared. Unfortunately, this may appear to invert
 the meaning of the flags from the way they are used in clone(2).
 However, there was no easy solution that was less confusing and that
 allowed incremental context unsharing in future without an ABI change.
 unshare interface should accommodate possible future addition of
 new context flags without requiring a rebuild of old applications.
 If and when new context flags are added, unshare design should allow
 incremental unsharing of those resources on an as needed basis.
 5) Functional Specification
 ---------------------------
 NAME
 	unshare - disassociate parts of the process execution context
 SYNOPSIS
 	#include <sched.h>
 	int unshare(int flags);
 DESCRIPTION
 	unshare allows a process to disassociate parts of its execution
 	context that are currently being shared with other processes. Part
 	of execution context, such as the namespace, is shared by default
 	when a new process is created using fork(2), while other parts,
 	such as the virtual memory, open file descriptors, etc, may be
 	shared by explicit request to share them when creating a process
 	using clone(2).
 	The main use of unshare is to allow a process to control its
 	shared execution context without creating a new process.
 	The flags argument specifies one or bitwise-or'ed of several of
 	the following constants.
 	CLONE_FS
 		If CLONE_FS is set, file system information of the caller
 		is disassociated from the shared file system information.
 	CLONE_FILES
 		If CLONE_FILES is set, the file descriptor table of the
 		caller is disassociated from the shared file descriptor
 		table.
 	CLONE_NEWNS
 		If CLONE_NEWNS is set, the namespace of the caller is
 		disassociated from the shared namespace.
 	CLONE_VM
 		If CLONE_VM is set, the virtual memory of the caller is
 		disassociated from the shared virtual memory.
 RETURN VALUE
 	On success, zero returned. On failure, -1 is returned and errno is
 ERRORS
 	EPERM	CLONE_NEWNS was specified by a non-root process (process
 		without CAP_SYS_ADMIN).
 	ENOMEM	Cannot allocate sufficient memory to copy parts of caller's
 		context that need to be unshared.
 	EINVAL	Invalid flag was specified as an argument.
 CONFORMING TO
 	The unshare() call is Linux-specific and  should  not be used
 	in programs intended to be portable.
 SEE ALSO
 	clone(2), fork(2)
 6) High Level Design
 --------------------
 Depending on the flags argument, the unshare system call allocates
 appropriate process context structures, populates it with values from
 the current shared version, associates newly duplicated structures
 with the current task structure and releases corresponding shared
 versions. Helper functions of clone (copy_*) could not be used
 directly by unshare because of the following two reasons.
  1) clone operates on a newly allocated not-yet-active task
     structure, where as unshare operates on the current active
     task. Therefore unshare has to take appropriate task_lock()
     before associating newly duplicated context structures
  2) unshare has to allocate and duplicate all context structures
     that are being unshared, before associating them with the
     current task and releasing older shared structures. Failure
     do so will create race conditions and/or oops when trying
     to backout due to an error. Consider the case of unsharing
     both virtual memory and namespace. After successfully unsharing
     vm, if the system call encounters an error while allocating
     new namespace structure, the error return code will have to
     reverse the unsharing of vm. As part of the reversal the
     system call will have to go back to older, shared, vm
     structure, which may not exist anymore.
 Therefore code from copy_* functions that allocated and duplicated
 current context structure was moved into new dup_* functions. Now,
 copy_* functions call dup_* functions to allocate and duplicate
 appropriate context structures and then associate them with the
 task structure that is being constructed. unshare system call on
 the other hand performs the following:
  1) Check flags to force missing, but implied, flags
  2) For each context structure, call the corresponding unshare
     helper function to allocate and duplicate a new context
     structure, if the appropriate bit is set in the flags argument.
  3) If there is no error in allocation and duplication and there
     are new context structures then lock the current task structure,
     associate new context structures with the current task structure,
     and release the lock on the current task structure.
  4) Appropriately release older, shared, context structures.
 7) Low Level Design
 -------------------
 Implementation of unshare can be grouped in the following 4 different
 items:
  a) Reorganization of existing copy_* functions
  b) unshare system call service function
  c) unshare helper functions for each different process context
  d) Registration of system call number for different architectures
  7.1) Reorganization of copy_* functions
       Each copy function such as copy_mm, copy_namespace, copy_files,
       etc, had roughly two components. The first component allocated
       and duplicated the appropriate structure and the second component
       linked it to the task structure passed in as an argument to the copy
       function. The first component was split into its own function.
       These dup_* functions allocated and duplicated the appropriate
       context structure. The reorganized copy_* functions invoked
       their corresponding dup_* functions and then linked the newly
       duplicated structures to the task structure with which the
       copy function was called.
  7.2) unshare system call service function
       * Check flags
 	 Force implied flags. If CLONE_THREAD is set force CLONE_VM.
 	 If CLONE_VM is set, force CLONE_SIGHAND. If CLONE_SIGHAND is
 	 set and signals are also being shared, force CLONE_THREAD. If
 	 CLONE_NEWNS is set, force CLONE_FS.
       * For each context flag, invoke the corresponding unshare_*
 	 helper routine with flags passed into the system call and a
 	 reference to pointer pointing the new unshared structure
       * If any new structures are created by unshare_* helper
 	 functions, take the task_lock() on the current task,
 	 modify appropriate context pointers, and release the
         task lock.
       * For all newly unshared structures, release the corresponding
         older, shared, structures.
  7.3) unshare_* helper functions
       For unshare_* helpers corresponding to CLONE_SYSVSEM, CLONE_SIGHAND,
       and CLONE_THREAD, return -EINVAL since they are not implemented yet.
       For others, check the flag value to see if the unsharing is
       required for that structure. If it is, invoke the corresponding
       dup_* function to allocate and duplicate the structure and return
       a pointer to it.
  7.4) Appropriately modify architecture specific code to register the
-       the new system call.
+       new system call.
 8) Test Specification
 ---------------------
 The test for unshare should test the following:
  1) Valid flags: Test to check that clone flags for signal and
 	signal handlers, for which unsharing is not implemented
 	yet, return -EINVAL.
  2) Missing/implied flags: Test to make sure that if unsharing
 	namespace without specifying unsharing of filesystem, correctly
 	unshares both namespace and filesystem information.
  3) For each of the four (namespace, filesystem, files and vm)
 	supported unsharing, verify that the system call correctly
 	unshares the appropriate structure. Verify that unsharing
 	them individually as well as in combination with each
 	other works as expected.
  4) Concurrent execution: Use shared memory segments and futex on
 	an address in the shm segment to synchronize execution of
 	about 10 threads. Have a couple of threads execute execve,
 	a couple _exit and the rest unshare with different combination
 	of flags. Verify that unsharing is performed as expected and
 	that there are no oops or hangs.
 9) Future Work
 --------------
 The current implementation of unshare does not allow unsharing of
 signals and signal handlers. Signals are complex to begin with and
 to unshare signals and/or signal handlers of a currently running
 process is even more complex. If in the future there is a specific
 need to allow unsharing of signals and/or signal handlers, it can
 be incrementally added to unshare without affecting legacy
 applications using unshare.
 Revised: 2004-Oct-21
 This is the documentation of (hopefully) all possible error codes (and
 their interpretation) that can be returned from usbcore.
 Some of them are returned by the Host Controller Drivers (HCDs), which
 device drivers only see through usbcore.  As a rule, all the HCDs should
 behave the same except for transfer speed dependent behaviors and the
 way certain faults are reported.
 **************************************************************************
 *                   Error codes returned by usb_submit_urb               *
 **************************************************************************
 Non-USB-specific:
 0		URB submission went fine
 -ENOMEM		no memory for allocation of internal structures	
 USB-specific:
 -ENODEV		specified USB-device or bus doesn't exist
 -ENOENT		specified interface or endpoint does not exist or
 		is not enabled
 -ENXIO		host controller driver does not support queuing of this type
 		of urb.  (treat as a host controller bug.)
 -EINVAL		a) Invalid transfer type specified (or not supported)
 		b) Invalid or unsupported periodic transfer interval
 		c) ISO: attempted to change transfer interval
 		d) ISO: number_of_packets is < 0
 		e) various other cases
 -EAGAIN		a) specified ISO start frame too early
 		b) (using ISO-ASAP) too much scheduled for the future
 		   wait some time and try again.
 -EFBIG		Host controller driver can't schedule that many ISO frames.
 -EPIPE		Specified endpoint is stalled.  For non-control endpoints,
 		reset this status with usb_clear_halt().
 -EMSGSIZE	(a) endpoint maxpacket size is zero; it is not usable
 		    in the current interface altsetting.
 		(b) ISO packet is larger than the endpoint maxpacket.
 		(c) requested data transfer length is invalid: negative
 		    or too large for the host controller.
 -ENOSPC		This request would overcommit the usb bandwidth reserved
 		for periodic transfers (interrupt, isochronous).
 -ESHUTDOWN	The device or host controller has been disabled due to some
 		problem that could not be worked around.
 -EPERM		Submission failed because urb->reject was set.
 -EHOSTUNREACH	URB was rejected because the device is suspended.
 **************************************************************************
 *                   Error codes returned by in urb->status               *
 *                   or in iso_frame_desc[n].status (for ISO)             *
 **************************************************************************
 USB device drivers may only test urb status values in completion handlers.
 This is because otherwise there would be a race between HCDs updating
 these values on one CPU, and device drivers testing them on another CPU.
 A transfer's actual_length may be positive even when an error has been
 reported.  That's because transfers often involve several packets, so that
 one or more packets could finish before an error stops further endpoint I/O.
 0			Transfer completed successfully
 -ENOENT			URB was synchronously unlinked by usb_unlink_urb
 -EINPROGRESS		URB still pending, no results yet
 			(That is, if drivers see this it's a bug.)
 -EPROTO (*, **)		a) bitstuff error
 			b) no response packet received within the
 			   prescribed bus turn-around time
 			c) unknown USB error 
 -EILSEQ (*, **)		a) CRC mismatch
 			b) no response packet received within the
 			   prescribed bus turn-around time
 			c) unknown USB error 
 			Note that often the controller hardware does not
 			distinguish among cases a), b), and c), so a
 			driver cannot tell whether there was a protocol
 			error, a failure to respond (often caused by
 			device disconnect), or some other fault.
 -ETIME (**)		No response packet received within the prescribed
 			bus turn-around time.  This error may instead be
 			reported as -EPROTO or -EILSEQ.
 -ETIMEDOUT		Synchronous USB message functions use this code
 			to indicate timeout expired before the transfer
 			completed, and no other error was reported by HC.
 -EPIPE (**)		Endpoint stalled.  For non-control endpoints,
 			reset this status with usb_clear_halt().
 -ECOMM			During an IN transfer, the host controller
 			received data from an endpoint faster than it
 			could be written to system memory
 -ENOSR			During an OUT transfer, the host controller
 			could not retrieve data from system memory fast
 			enough to keep up with the USB data rate
 -EOVERFLOW (*)		The amount of data returned by the endpoint was
 			greater than either the max packet size of the
 			endpoint or the remaining buffer size.  "Babble".
 -EREMOTEIO		The data read from the endpoint did not fill the
 			specified buffer, and URB_SHORT_NOT_OK was set in
 			urb->transfer_flags.
 -ENODEV			Device was removed.  Often preceded by a burst of
 			other errors, since the hub driver doesn't detect
 			device removal events immediately.
 -EXDEV			ISO transfer only partially completed
 			look at individual frame status for details
 -EINVAL			ISO madness, if this happens: Log off and go home
 -ECONNRESET		URB was asynchronously unlinked by usb_unlink_urb
 -ESHUTDOWN		The device or host controller has been disabled due
 			to some problem that could not be worked around,
 			such as a physical disconnect.
 (*) Error codes like -EPROTO, -EILSEQ and -EOVERFLOW normally indicate
 hardware problems such as bad devices (including firmware) or cables.
 (**) This is also one of several codes that different kinds of host
-controller use to to indicate a transfer has failed because of device
+controller use to indicate a transfer has failed because of device
 disconnect.  In the interval before the hub driver starts disconnect
 processing, devices may receive such fault reports for every request.
 **************************************************************************
 *              Error codes returned by usbcore-functions                 *
 *           (expect also other submit and transfer status codes)         *
 **************************************************************************
 usb_register():
 -EINVAL			error during registering new driver
 usb_get_*/usb_set_*():
 usb_control_msg():
 usb_bulk_msg():
 -ETIMEDOUT		Timeout expired before the transfer completed.
 Care and feeding of your Human Interface Devices
 INTRODUCTION
 In addition to the normal input type HID devices, USB also uses the
 human interface device protocols for things that are not really human
 interfaces, but have similar sorts of communication needs. The two big
 examples for this are power devices (especially uninterruptable power
 supplies) and monitor control on higher end monitors.
 To support these disparite requirements, the Linux USB system provides
 HID events to two separate interfaces:
 * the input subsystem, which converts HID events into normal input
 device interfaces (such as keyboard, mouse and joystick) and a
 normalised event interface - see Documentation/input/input.txt
 * the hiddev interface, which provides fairly raw HID events
 The data flow for a HID event produced by a device is something like
 the following :
 usb.c ---> hid-core.c  ----> hid-input.c ----> [keyboard/mouse/joystick/event]
                         |
                         |
                          --> hiddev.c ----> POWER / MONITOR CONTROL 
 In addition, other subsystems (apart from USB) can potentially feed
 events into the input subsystem, but these have no effect on the hid
 device interface.
 USING THE HID DEVICE INTERFACE
 The hiddev interface is a char interface using the normal USB major,
 with the minor numbers starting at 96 and finishing at 111. Therefore,
 you need the following commands:
 mknod /dev/usb/hiddev0 c 180 96
 mknod /dev/usb/hiddev1 c 180 97
 mknod /dev/usb/hiddev2 c 180 98
 mknod /dev/usb/hiddev3 c 180 99
 mknod /dev/usb/hiddev4 c 180 100
 mknod /dev/usb/hiddev5 c 180 101
 mknod /dev/usb/hiddev6 c 180 102
 mknod /dev/usb/hiddev7 c 180 103
 mknod /dev/usb/hiddev8 c 180 104
 mknod /dev/usb/hiddev9 c 180 105
 mknod /dev/usb/hiddev10 c 180 106
 mknod /dev/usb/hiddev11 c 180 107
 mknod /dev/usb/hiddev12 c 180 108
 mknod /dev/usb/hiddev13 c 180 109
 mknod /dev/usb/hiddev14 c 180 110
 mknod /dev/usb/hiddev15 c 180 111
 So you point your hiddev compliant user-space program at the correct
 interface for your device, and it all just works.
 Assuming that you have a hiddev compliant user-space program, of
 course. If you need to write one, read on.
 THE HIDDEV API
 This description should be read in conjunction with the HID
 specification, freely available from http://www.usb.org, and
 conveniently linked of http://www.linux-usb.org.
 The hiddev API uses a read() interface, and a set of ioctl() calls.
 HID devices exchange data with the host computer using data
 bundles called "reports".  Each report is divided into "fields",
 each of which can have one or more "usages".  In the hid-core,
 each one of these usages has a single signed 32 bit value.
 read():
 This is the event interface.  When the HID device's state changes,
 it performs an interrupt transfer containing a report which contains
 the changed value.  The hid-core.c module parses the report, and
 returns to hiddev.c the individual usages that have changed within
 the report.  In its basic mode, the hiddev will make these individual
 usage changes available to the reader using a struct hiddev_event:
       struct hiddev_event {
           unsigned hid;
           signed int value;
       };
 containing the HID usage identifier for the status that changed, and
 the value that it was changed to. Note that the structure is defined
 within <linux/hiddev.h>, along with some other useful #defines and
 structures.  The HID usage identifier is a composite of the HID usage
 page shifted to the 16 high order bits ORed with the usage code.  The
 behavior of the read() function can be modified using the HIDIOCSFLAG
 ioctl() described below.
 ioctl(): 
 This is the control interface. There are a number of controls: 
 HIDIOCGVERSION - int (read)
 Gets the version code out of the hiddev driver.
 HIDIOCAPPLICATION - (none)
 This ioctl call returns the HID application usage associated with the
 hid device. The third argument to ioctl() specifies which application
 index to get. This is useful when the device has more than one
 application collection. If the index is invalid (greater or equal to
 the number of application collections this device has) the ioctl
 returns -1. You can find out beforehand how many application
 collections the device has from the num_applications field from the
 hiddev_devinfo structure. 
 HIDIOCGCOLLECTIONINFO - struct hiddev_collection_info (read/write)
 This returns a superset of the information above, providing not only
 application collections, but all the collections the device has.  It
 also returns the level the collection lives in the hierarchy.
 The user passes in a hiddev_collection_info struct with the index 
 field set to the index that should be returned.  The ioctl fills in 
 the other fields.  If the index is larger than the last collection 
 index, the ioctl returns -1 and sets errno to -EINVAL.
 HIDIOCGDEVINFO - struct hiddev_devinfo (read)
 Gets a hiddev_devinfo structure which describes the device.
-HIDIOCGSTRING - struct struct hiddev_string_descriptor (read/write)
+HIDIOCGSTRING - struct hiddev_string_descriptor (read/write)
 Gets a string descriptor from the device. The caller must fill in the
 "index" field to indicate which descriptor should be returned.
 HIDIOCINITREPORT - (none)
 Instructs the kernel to retrieve all input and feature report values
 from the device. At this point, all the usage structures will contain
 current values for the device, and will maintain it as the device
 changes.  Note that the use of this ioctl is unnecessary in general,
 since later kernels automatically initialize the reports from the
 device at attach time.
 HIDIOCGNAME - string (variable length)
 Gets the device name
 HIDIOCGREPORT - struct hiddev_report_info (write)
 Instructs the kernel to get a feature or input report from the device,
 in order to selectively update the usage structures (in contrast to
 INITREPORT).
 HIDIOCSREPORT - struct hiddev_report_info (write)
 Instructs the kernel to send a report to the device. This report can
 be filled in by the user through HIDIOCSUSAGE calls (below) to fill in
 individual usage values in the report before sending the report in full
 to the device. 
 HIDIOCGREPORTINFO - struct hiddev_report_info (read/write)
 Fills in a hiddev_report_info structure for the user. The report is
 looked up by type (input, output or feature) and id, so these fields
 must be filled in by the user. The ID can be absolute -- the actual
 report id as reported by the device -- or relative --
 HID_REPORT_ID_FIRST for the first report, and (HID_REPORT_ID_NEXT |
 report_id) for the next report after report_id. Without a-priori
 information about report ids, the right way to use this ioctl is to
 use the relative IDs above to enumerate the valid IDs. The ioctl
 returns non-zero when there is no more next ID. The real report ID is
 filled into the returned hiddev_report_info structure. 
 HIDIOCGFIELDINFO - struct hiddev_field_info (read/write)
 Returns the field information associated with a report in a
 hiddev_field_info structure. The user must fill in report_id and
 report_type in this structure, as above. The field_index should also
 be filled in, which should be a number from 0 and maxfield-1, as
 returned from a previous HIDIOCGREPORTINFO call. 
 HIDIOCGUCODE - struct hiddev_usage_ref (read/write)
 Returns the usage_code in a hiddev_usage_ref structure, given that
 given its report type, report id, field index, and index within the
 field have already been filled into the structure.
 HIDIOCGUSAGE - struct hiddev_usage_ref (read/write)
 Returns the value of a usage in a hiddev_usage_ref structure. The
 usage to be retrieved can be specified as above, or the user can
 choose to fill in the report_type field and specify the report_id as
 HID_REPORT_ID_UNKNOWN. In this case, the hiddev_usage_ref will be
 filled in with the report and field information associated with this
 usage if it is found. 
 HIDIOCSUSAGE - struct hiddev_usage_ref (write)
 Sets the value of a usage in an output report.  The user fills in
 the hiddev_usage_ref structure as above, but additionally fills in
 the value field.
 HIDIOGCOLLECTIONINDEX - struct hiddev_usage_ref (write)
 Returns the collection index associated with this usage.  This
 indicates where in the collection hierarchy this usage sits.
 HIDIOCGFLAG - int (read)
 HIDIOCSFLAG - int (write)
 These operations respectively inspect and replace the mode flags
 that influence the read() call above.  The flags are as follows:
    HIDDEV_FLAG_UREF - read() calls will now return 
        struct hiddev_usage_ref instead of struct hiddev_event.
        This is a larger structure, but in situations where the
        device has more than one usage in its reports with the
        same usage code, this mode serves to resolve such
        ambiguity.
    HIDDEV_FLAG_REPORT - This flag can only be used in conjunction
        with HIDDEV_FLAG_UREF.  With this flag set, when the device
        sends a report, a struct hiddev_usage_ref will be returned
        to read() filled in with the report_type and report_id, but 
        with field_index set to FIELD_INDEX_NONE.  This serves as
        additional notification when the device has sent a report.
 INTRODUCTION
  The USB serial driver currently supports a number of different USB to
  serial converter products, as well as some devices that use a serial
  interface from userspace to talk to the device.
  See the individual product section below for specific information about
  the different devices.
 CONFIGURATION
  Currently the driver can handle up to 256 different serial interfaces at
  one time. 
    The major number that the driver uses is 188 so to use the driver,
    create the following nodes:
 	mknod /dev/ttyUSB0 c 188 0
 	mknod /dev/ttyUSB1 c 188 1
 	mknod /dev/ttyUSB2 c 188 2
 	mknod /dev/ttyUSB3 c 188 3
 		.
 		.
 		.
 	mknod /dev/ttyUSB254 c 188 254
 	mknod /dev/ttyUSB255 c 188 255
  When the device is connected and recognized by the driver, the driver
  will print to the system log, which node(s) the device has been bound
  to.
 SPECIFIC DEVICES SUPPORTED
 ConnectTech WhiteHEAT 4 port converter
  ConnectTech has been very forthcoming with information about their
  device, including providing a unit to test with.
  The driver is officially supported by Connect Tech Inc.
  http://www.connecttech.com
  For any questions or problems with this driver, please contact
  Stuart MacDonald at stuartm@connecttech.com
 HandSpring Visor, Palm USB, and Clié USB driver
  This driver works with all HandSpring USB, Palm USB, and Sony Clié USB
  devices.
  Only when the device tries to connect to the host, will the device show
  up to the host as a valid USB device. When this happens, the device is
  properly enumerated, assigned a port, and then communication _should_ be
  possible. The driver cleans up properly when the device is removed, or
  the connection is canceled on the device.
  NOTE:
    This means that in order to talk to the device, the sync button must be
    pressed BEFORE trying to get any program to communicate to the device.
    This goes against the current documentation for pilot-xfer and other
    packages, but is the only way that it will work due to the hardware
    in the device.
  When the device is connected, try talking to it on the second port
  (this is usually /dev/ttyUSB1 if you do not have any other usb-serial
  devices in the system.) The system log should tell you which port is
  the port to use for the HotSync transfer. The "Generic" port can be used
  for other device communication, such as a PPP link.
  For some Sony Clié devices, /dev/ttyUSB0 must be used to talk to the
  device.  This is true for all OS version 3.5 devices, and most devices
  that have had a flash upgrade to a newer version of the OS.  See the
  kernel system log for information on which is the correct port to use.
  If after pressing the sync button, nothing shows up in the system log,
  try resetting the device, first a hot reset, and then a cold reset if
  necessary.  Some devices need this before they can talk to the USB port
  properly.
  Devices that are not compiled into the kernel can be specified with module
  parameters.  e.g. modprobe visor vendor=0x54c product=0x66
  There is a webpage and mailing lists for this portion of the driver at:
  http://usbvisor.sourceforge.net/
  For any questions or problems with this driver, please contact Greg
  Kroah-Hartman at greg@kroah.com
 PocketPC PDA Driver
  This driver can be used to connect to Compaq iPAQ, HP Jornada, Casio EM500
  and other PDAs running Windows CE 3.0 or PocketPC 2002 using a USB
  cable/cradle.
  Most devices supported by ActiveSync are supported out of the box.
  For others, please use module parameters to specify the product and vendor
  id. e.g. modprobe ipaq vendor=0x3f0 product=0x1125
  The driver presents a serial interface (usually on /dev/ttyUSB0) over
  which one may run ppp and establish a TCP/IP link to the PDA. Once this
  is done, you can transfer files, backup, download email etc. The most
  significant advantage of using USB is speed - I can get 73 to 113
  kbytes/sec for download/upload to my iPAQ.
  This driver is only one of a set of components required to utilize
  the USB connection. Please visit http://synce.sourceforge.net which
  contains the necessary packages and a simple step-by-step howto.
  Once connected, you can use Win CE programs like ftpView, Pocket Outlook
  from the PDA and xcerdisp, synce utilities from the Linux side.
  To use Pocket IE, follow the instructions given at
  http://www.tekguru.co.uk/EM500/usbtonet.htm to achieve the same thing
  on Win98. Omit the proxy server part; Linux is quite capable of forwarding
  packets unlike Win98. Another modification is required at least for the
  iPAQ - disable autosync by going to the Start/Settings/Connections menu
  and unchecking the "Automatically synchronize ..." box. Go to
  Start/Programs/Connections, connect the cable and select "usbdial" (or
  whatever you named your new USB connection). You should finally wind
  up with a "Connected to usbdial" window with status shown as connected.
  Now start up PIE and browse away.
  If it doesn't work for some reason, load both the usbserial and ipaq module
  with the module parameter "debug" set to 1 and examine the system log.
  You can also try soft-resetting your PDA before attempting a connection.
  Other functionality may be possible depending on your PDA. According to
  Wes Cilldhaire <billybobjoehenrybob@hotmail.com>, with the Toshiba E570,
  ...if you boot into the bootloader (hold down the power when hitting the
  reset button, continuing to hold onto the power until the bootloader screen
  is displayed), then put it in the cradle with the ipaq driver loaded, open
  a terminal on /dev/ttyUSB0, it gives you a "USB Reflash" terminal, which can
  be used to flash the ROM, as well as the microP code..  so much for needing
  Toshiba's $350 serial cable for flashing!! :D
  NOTE: This has NOT been tested. Use at your own risk.
  For any questions or problems with the driver, please contact Ganesh
  Varadarajan <ganesh@veritas.com>
 Keyspan PDA Serial Adapter
  Single port DB-9 serial adapter, pushed as a PDA adapter for iMacs (mostly
  sold in Macintosh catalogs, comes in a translucent white/green dongle).
  Fairly simple device. Firmware is homebrew.
  This driver also works for the Xircom/Entrgra single port serial adapter.
  Current status:
   Things that work:
     basic input/output (tested with 'cu')
     blocking write when serial line can't keep up
     changing baud rates (up to 115200)
     getting/setting modem control pins (TIOCM{GET,SET,BIS,BIC})
     sending break (although duration looks suspect)
   Things that don't:
     device strings (as logged by kernel) have trailing binary garbage
     device ID isn't right, might collide with other Keyspan products
     changing baud rates ought to flush tx/rx to avoid mangled half characters
   Big Things on the todo list:
     parity, 7 vs 8 bits per char, 1 or 2 stop bits
     HW flow control
     not all of the standard USB descriptors are handled: Get_Status, Set_Feature
     O_NONBLOCK, select()
  For any questions or problems with this driver, please contact Brian
  Warner at warner@lothar.com 
 Keyspan USA-series Serial Adapters
  Single, Dual and Quad port adapters - driver uses Keyspan supplied 
  firmware and is being developed with their support.
  Current status:
    The USA-18X, USA-28X, USA-19, USA-19W and USA-49W are supported and
    have been pretty throughly tested at various baud rates with 8-N-1
    character settings.  Other character lengths and parity setups are
    presently untested.
    The USA-28 isn't yet supported though doing so should be pretty
    straightforward.  Contact the maintainer if you require this
    functionality.
  More information is available at:
        http://misc.nu/hugh/keyspan.html
  For any questions or problems with this driver, please contact Hugh
  Blemings at hugh@misc.nu
 FTDI Single Port Serial Driver
  This is a single port DB-25 serial adapter. More information about this
  device and the Linux driver can be found at:
 	http://reality.sgi.com/bryder_wellington/ftdi_sio/
  For any questions or problems with this driver, please contact Bill Ryder
  at bryder@sgi.com
 ZyXEL omni.net lcd plus ISDN TA
  This is an ISDN TA. Please report both successes and troubles to
  azummo@towertech.it
 Cypress M8 CY4601 Family Serial Driver
  This driver was in most part developed by Neil "koyama" Whelchel.  It
  has been improved since that previous form to support dynamic serial
  line settings and improved line handling.  The driver is for the most
  part stable and has been tested on an smp machine. (dual p2)
    Chipsets supported under CY4601 family:
 		CY7C63723, CY7C63742, CY7C63743, CY7C64013
    Devices supported:
 		-DeLorme's USB Earthmate (SiRF Star II lp arch)
 		-Cypress HID->COM RS232 adapter
 		Note: Cypress Semiconductor claims no affiliation with the
-			the hid->com device.
+			hid->com device.
 	Most devices using chipsets under the CY4601 family should
     work with the driver.  As long as they stay true to the CY4601
     usbserial specification.
    Technical notes:
        The Earthmate starts out at 4800 8N1 by default... the driver will
 	upon start init to this setting.  usbserial core provides the rest
 	of the termios settings, along with some custom termios so that the
 	output is in proper format and parsable.
 	The device can be put into sirf mode by issuing NMEA command:
 		$PSRF100,<protocol>,<baud>,<databits>,<stopbits>,<parity>*CHECKSUM
 		$PSRF100,0,9600,8,1,0*0C
 		It should then be sufficient to change the port termios to match this
 		to begin communicating.
 	As far as I can tell it supports pretty much every sirf command as
 	documented online available with firmware 2.31, with some unknown
 	message ids.
 	The hid->com adapter can run at a maximum baud of 115200bps.  Please note
 	that the device has trouble or is incapable of raising line voltage properly.
 	It will be fine with null modem links, as long as you do not try to link two
 	together without hacking the adapter to set the line high.
 	The driver is smp safe.  Performance with the driver is rather low when using
 	it for transfering files.  This is being worked on, but I would be willing to
 	accept patches.  An urb queue or packet buffer would likely fit the bill here.
 	If you have any questions, problems, patches, feature requests, etc. you can
 	contact me here via email:
 					dignome@gmail.com
 		(your problems/patches can alternately be submitted to usb-devel)
 Digi AccelePort Driver
  This driver supports the Digi AccelePort USB 2 and 4 devices, 2 port
  (plus a parallel port) and 4 port USB serial converters.  The driver
  does NOT yet support the Digi AccelePort USB 8.
  This driver works under SMP with the usb-uhci driver.  It does not
  work under SMP with the uhci driver.
  The driver is generally working, though we still have a few more ioctls
  to implement and final testing and debugging to do.  The parallel port
  on the USB 2 is supported as a serial to parallel converter; in other
  words, it appears as another USB serial port on Linux, even though
  physically it is really a parallel port.  The Digi Acceleport USB 8
  is not yet supported.
  Please contact Peter Berger (pberger@brimson.com) or Al Borchers
  (alborchers@steinerpoint.com) for questions or problems with this
  driver.
 Belkin USB Serial Adapter F5U103
  Single port DB-9/PS-2 serial adapter from Belkin with firmware by eTEK Labs.
  The Peracom single port serial adapter also works with this driver, as
  well as the GoHubs adapter.
  Current status:
    The following have been tested and work:
      Baud rate    300-230400               
      Data bits    5-8
      Stop bits    1-2
      Parity       N,E,O,M,S
      Handshake    None, Software (XON/XOFF), Hardware (CTSRTS,CTSDTR)*
      Break        Set and clear
      Line contrl  Input/Output query and control **
      *  Hardware input flow control is only enabled for firmware
         levels above 2.06.  Read source code comments describing Belkin
         firmware errata.  Hardware output flow control is working for all
         firmware versions.
      ** Queries of inputs (CTS,DSR,CD,RI) show the last
         reported state.  Queries of outputs (DTR,RTS) show the last
         requested state and may not reflect current state as set by
         automatic hardware flow control.
  TO DO List:
    -- Add true modem contol line query capability.  Currently tracks the
       states reported by the interrupt and the states requested.
    -- Add error reporting back to application for UART error conditions.
    -- Add support for flush ioctls.
    -- Add everything else that is missing :)
  For any questions or problems with this driver, please contact William
  Greathouse at wgreathouse@smva.com
 Empeg empeg-car Mark I/II Driver
  This is an experimental driver to provide connectivity support for the
  client synchronization tools for an Empeg empeg-car mp3 player.
  Tips:
    * Don't forget to create the device nodes for ttyUSB{0,1,2,...}
    * modprobe empeg (modprobe is your friend)
    * emptool --usb /dev/ttyUSB0 (or whatever you named your device node)
  For any questions or problems with this driver, please contact Gary
  Brubaker at xavyer@ix.netcom.com
 MCT USB Single Port Serial Adapter U232
  This driver is for the MCT USB-RS232 Converter (25 pin, Model No.
  U232-P25) from Magic Control Technology Corp. (there is also a 9 pin
  Model No. U232-P9). More information about this device can be found at
  the manufacture's web-site: http://www.mct.com.tw.
  The driver is generally working, though it still needs some more testing.
  It is derived from the Belkin USB Serial Adapter F5U103 driver and its
  TODO list is valid for this driver as well.
  This driver has also been found to work for other products, which have
  the same Vendor ID but different Product IDs. Sitecom's U232-P25 serial
  converter uses Product ID 0x230 and Vendor ID 0x711 and works with this
  driver. Also, D-Link's DU-H3SP USB BAY also works with this driver.
  For any questions or problems with this driver, please contact Wolfgang
  Grandegger at wolfgang@ces.ch
 Inside Out Networks Edgeport Driver
  This driver supports all devices made by Inside Out Networks, specifically
  the following models:
       Edgeport/4
       Rapidport/4
       Edgeport/4t
       Edgeport/2
       Edgeport/4i
       Edgeport/2i
       Edgeport/421
       Edgeport/21
       Edgeport/8
       Edgeport/8 Dual
       Edgeport/2D8
       Edgeport/4D8
       Edgeport/8i
       Edgeport/2 DIN
       Edgeport/4 DIN
       Edgeport/16 Dual
  For any questions or problems with this driver, please contact Greg
  Kroah-Hartman at greg@kroah.com
 REINER SCT cyberJack pinpad/e-com USB chipcard reader
  Interface to ISO 7816 compatible contactbased chipcards, e.g. GSM SIMs.
  Current status:
    This is the kernel part of the driver for this USB card reader.
    There is also a user part for a CT-API driver available. A site
    for downloading is TBA. For now, you can request it from the
    maintainer (linux-usb@sii.li).
  For any questions or problems with this driver, please contact
  linux-usb@sii.li
 Prolific PL2303 Driver
  This driver supports any device that has the PL2303 chip from Prolific
  in it.  This includes a number of single port USB to serial
  converters and USB GPS devices.  Devices from Aten (the UC-232) and
  IO-Data work with this driver, as does the DCU-11 mobile-phone cable.
  For any questions or problems with this driver, please contact Greg
  Kroah-Hartman at greg@kroah.com
 KL5KUSB105 chipset / PalmConnect USB single-port adapter
 Current status:
  The driver was put together by looking at the usb bus transactions
  done by Palm's driver under Windows, so a lot of functionality is
  still missing.  Notably, serial ioctls are sometimes faked or not yet
  implemented.  Support for finding out about DSR and CTS line status is
  however implemented (though not nicely), so your favorite autopilot(1)
  and pilot-manager -daemon calls will work.  Baud rates up to 115200
  are supported, but handshaking (software or hardware) is not, which is
  why it is wise to cut down on the rate used is wise for large
  transfers until this is settled.
 Options supported:
  If this driver is compiled as a module you can pass the following
  options to it:
  debug			- extra verbose debugging info
  			  (default: 0; nonzero enables)
  use_lowlatency	- use low_latency flag to speed up tty layer
-			  when reading from from the device.
+			  when reading from the device.
 			  (default: 0; nonzero enables)
  See http://www.uuhaus.de/linux/palmconnect.html for up-to-date
  information on this driver.
 AIRcable USB Dongle Bluetooth driver
  If there is the cdc_acm driver loaded in the system, you will find that the
  cdc_acm claims the device before AIRcable can. This is simply corrected
  by unloading both modules and then loading the aircable module before
  cdc_acm module
 Generic Serial driver
  If your device is not one of the above listed devices, compatible with
  the above models, you can try out the "generic" interface. This
  interface does not provide any type of control messages sent to the
  device, and does not support any kind of device flow control. All that
  is required of your device is that it has at least one bulk in endpoint,
  or one bulk out endpoint. 
  To enable the generic driver to recognize your device, build the driver
  as a module and load it by the following invocation:
 	insmod usbserial vendor=0x#### product=0x####
  where the #### is replaced with the hex representation of your device's
  vendor id and product id.
  This driver has been successfully used to connect to the NetChip USB
  development board, providing a way to develop USB firmware without
  having to write a custom driver.
  For any questions or problems with this driver, please contact Greg
  Kroah-Hartman at greg@kroah.com
 CONTACT:
  If anyone has any problems using these drivers, with any of the above
  specified products, please contact the specific driver's author listed
  above, or join the Linux-USB mailing list (information on joining the
  mailing list, as well as a link to its searchable archive is at
  http://www.linux-usb.org/ )
 Greg Kroah-Hartman
 greg@kroah.com
 $Id$
 Mike Isely <isely@pobox.com>
 			    pvrusb2 driver
 Background:
  This driver is intended for the "Hauppauge WinTV PVR USB 2.0", which
  is a USB 2.0 hosted TV Tuner.  This driver is a work in progress.
  Its history started with the reverse-engineering effort by Björn
  Danielsson <pvrusb2@dax.nu> whose web page can be found here:
    http://pvrusb2.dax.nu/
  From there Aurelien Alleaume <slts@free.fr> began an effort to
  create a video4linux compatible driver.  I began with Aurelien's
  last known snapshot and evolved the driver to the state it is in
  here.
  More information on this driver can be found at:
    http://www.isely.net/pvrusb2.html
  This driver has a strong separation of layers.  They are very
  roughly:
  1a. Low level wire-protocol implementation with the device.
  1b. I2C adaptor implementation and corresponding I2C client drivers
      implemented elsewhere in V4L.
  1c. High level hardware driver implementation which coordinates all
      activities that ensure correct operation of the device.
  2.  A "context" layer which manages instancing of driver, setup,
      tear-down, arbitration, and interaction with high level
      interfaces appropriately as devices are hotplugged in the
      system.
  3.  High level interfaces which glue the driver to various published
      Linux APIs (V4L, sysfs, maybe DVB in the future).
  The most important shearing layer is between the top 2 layers.  A
  lot of work went into the driver to ensure that any kind of
  conceivable API can be laid on top of the core driver.  (Yes, the
  driver internally leverages V4L to do its work but that really has
  nothing to do with the API published by the driver to the outside
  world.)  The architecture allows for different APIs to
  simultaneously access the driver.  I have a strong sense of fairness
  about APIs and also feel that it is a good design principle to keep
  implementation and interface isolated from each other.  Thus while
  right now the V4L high level interface is the most complete, the
  sysfs high level interface will work equally well for similar
  functions, and there's no reason I see right now why it shouldn't be
  possible to produce a DVB high level interface that can sit right
  alongside V4L.
  NOTE: Complete documentation on the pvrusb2 driver is contained in
  the html files within the doc directory; these are exactly the same
  as what is on the web site at the time.  Browse those files
  (especially the FAQ) before asking questions.
 Building
  To build these modules essentially amounts to just running "Make",
  but you need the kernel source tree nearby and you will likely also
  want to set a few controlling environment variables first in order
  to link things up with that source tree.  Please see the Makefile
  here for comments that explain how to do that.
 Source file list / functional overview:
  (Note: The term "module" used below generally refers to loosely
  defined functional units within the pvrusb2 driver and bears no
  relation to the Linux kernel's concept of a loadable module.)
  pvrusb2-audio.[ch] - This is glue logic that resides between this
    driver and the msp3400.ko I2C client driver (which is found
    elsewhere in V4L).
  pvrusb2-context.[ch] - This module implements the context for an
    instance of the driver.  Everything else eventually ties back to
    or is otherwise instanced within the data structures implemented
    here.  Hotplugging is ultimately coordinated here.  All high level
    interfaces tie into the driver through this module.  This module
    helps arbitrate each interface's access to the actual driver core,
    and is designed to allow concurrent access through multiple
    instances of multiple interfaces (thus you can for example change
    the tuner's frequency through sysfs while simultaneously streaming
    video through V4L out to an instance of mplayer).
  pvrusb2-debug.h - This header defines a printk() wrapper and a mask
    of debugging bit definitions for the various kinds of debug
    messages that can be enabled within the driver.
  pvrusb2-debugifc.[ch] - This module implements a crude command line
    oriented debug interface into the driver.  Aside from being part
    of the process for implementing manual firmware extraction (see
    the pvrusb2 web site mentioned earlier), probably I'm the only one
    who has ever used this.  It is mainly a debugging aid.
  pvrusb2-eeprom.[ch] - This is glue logic that resides between this
    driver the tveeprom.ko module, which is itself implemented
    elsewhere in V4L.
  pvrusb2-encoder.[ch] - This module implements all protocol needed to
    interact with the Conexant mpeg2 encoder chip within the pvrusb2
    device.  It is a crude echo of corresponding logic in ivtv,
    however the design goals (strict isolation) and physical layer
    (proxy through USB instead of PCI) are enough different that this
    implementation had to be completely different.
  pvrusb2-hdw-internal.h - This header defines the core data structure
    in the driver used to track ALL internal state related to control
    of the hardware.  Nobody outside of the core hardware-handling
    modules should have any business using this header.  All external
    access to the driver should be through one of the high level
    interfaces (e.g. V4L, sysfs, etc), and in fact even those high
    level interfaces are restricted to the API defined in
    pvrusb2-hdw.h and NOT this header.
  pvrusb2-hdw.h - This header defines the full internal API for
    controlling the hardware.  High level interfaces (e.g. V4L, sysfs)
    will work through here.
  pvrusb2-hdw.c - This module implements all the various bits of logic
    that handle overall control of a specific pvrusb2 device.
    (Policy, instantiation, and arbitration of pvrusb2 devices fall
    within the jurisdiction of pvrusb-context not here).
  pvrusb2-i2c-chips-*.c - These modules implement the glue logic to
    tie together and configure various I2C modules as they attach to
    the I2C bus.  There are two versions of this file.  The "v4l2"
    version is intended to be used in-tree alongside V4L, where we
    implement just the logic that makes sense for a pure V4L
    environment.  The "all" version is intended for use outside of
    V4L, where we might encounter other possibly "challenging" modules
    from ivtv or older kernel snapshots (or even the support modules
    in the standalone snapshot).
  pvrusb2-i2c-cmd-v4l1.[ch] - This module implements generic V4L1
    compatible commands to the I2C modules.  It is here where state
    changes inside the pvrusb2 driver are translated into V4L1
    commands that are in turn send to the various I2C modules.
  pvrusb2-i2c-cmd-v4l2.[ch] - This module implements generic V4L2
    compatible commands to the I2C modules.  It is here where state
    changes inside the pvrusb2 driver are translated into V4L2
    commands that are in turn send to the various I2C modules.
  pvrusb2-i2c-core.[ch] - This module provides an implementation of a
    kernel-friendly I2C adaptor driver, through which other external
    I2C client drivers (e.g. msp3400, tuner, lirc) may connect and
-    operate corresponding chips within the the pvrusb2 device.  It is
+    operate corresponding chips within the pvrusb2 device.  It is
    through here that other V4L modules can reach into this driver to
    operate specific pieces (and those modules are in turn driven by
    glue logic which is coordinated by pvrusb2-hdw, doled out by
    pvrusb2-context, and then ultimately made available to users
    through one of the high level interfaces).
  pvrusb2-io.[ch] - This module implements a very low level ring of
    transfer buffers, required in order to stream data from the
    device.  This module is *very* low level.  It only operates the
    buffers and makes no attempt to define any policy or mechanism for
    how such buffers might be used.
  pvrusb2-ioread.[ch] - This module layers on top of pvrusb2-io.[ch]
    to provide a streaming API usable by a read() system call style of
    I/O.  Right now this is the only layer on top of pvrusb2-io.[ch],
    however the underlying architecture here was intended to allow for
    other styles of I/O to be implemented with additonal modules, like
    mmap()'ed buffers or something even more exotic.
  pvrusb2-main.c - This is the top level of the driver.  Module level
    and USB core entry points are here.  This is our "main".
  pvrusb2-sysfs.[ch] - This is the high level interface which ties the
    pvrusb2 driver into sysfs.  Through this interface you can do
    everything with the driver except actually stream data.
  pvrusb2-tuner.[ch] - This is glue logic that resides between this
    driver and the tuner.ko I2C client driver (which is found
    elsewhere in V4L).
  pvrusb2-util.h - This header defines some common macros used
    throughout the driver.  These macros are not really specific to
    the driver, but they had to go somewhere.
  pvrusb2-v4l2.[ch] - This is the high level interface which ties the
    pvrusb2 driver into video4linux.  It is through here that V4L
    applications can open and operate the driver in the usual V4L
    ways.  Note that **ALL** V4L functionality is published only
    through here and nowhere else.
  pvrusb2-video-*.[ch] - This is glue logic that resides between this
    driver and the saa711x.ko I2C client driver (which is found
    elsewhere in V4L).  Note that saa711x.ko used to be known as
    saa7115.ko in ivtv.  There are two versions of this; one is
    selected depending on the particular saa711[5x].ko that is found.
  pvrusb2.h - This header contains compile time tunable parameters
    (and at the moment the driver has very little that needs to be
    tuned).
  -Mike Isely
  isely@pobox.com
 Frequently Asked Questions:
 ===========================
 subject: unified zoran driver (zr360x7, zoran, buz, dc10(+), dc30(+), lml33)
 website: http://mjpeg.sourceforge.net/driver-zoran/
 1. What cards are supported
 1.1 What the TV decoder can do an what not
 1.2 What the TV encoder can do an what not
 2. How do I get this damn thing to work
 3. What mainboard should I use (or why doesn't my card work)
 4. Programming interface
 5. Applications
 6. Concerning buffer sizes, quality, output size etc.
 7. It hangs/crashes/fails/whatevers! Help!
 8. Maintainers/Contacting
 9. License
 ===========================
 1. What cards are supported
 Iomega Buz, Linux Media Labs LML33/LML33R10, Pinnacle/Miro
 DC10/DC10+/DC30/DC30+ and related boards (available under various names).
 Iomega Buz:
 * Zoran zr36067 PCI controller
 * Zoran zr36060 MJPEG codec
 * Philips saa7111 TV decoder
 * Philips saa7185 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, saa7111, saa7185, zr36060, zr36067
 Inputs/outputs: Composite and S-video
 Norms: PAL, SECAM (720x576 @ 25 fps), NTSC (720x480 @ 29.97 fps)
 Card number: 7
 AverMedia 6 Eyes AVS6EYES:
 * Zoran zr36067 PCI controller
 * Zoran zr36060 MJPEG codec
 * Samsung ks0127 TV decoder
 * Conexant bt866  TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, ks0127, bt866, zr36060, zr36067
 Inputs/outputs: Six physical inputs. 1-6 are composite,
 		1-2, 3-4, 5-6 doubles as S-video,
 		1-3 triples as component.
 		One composite output.
 Norms: PAL, SECAM (720x576 @ 25 fps), NTSC (720x480 @ 29.97 fps)
 Card number: 8
 Not autodetected, card=8 is necessary.
 Linux Media Labs LML33:
 * Zoran zr36067 PCI controller
 * Zoran zr36060 MJPEG codec
 * Brooktree bt819 TV decoder
 * Brooktree bt856 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, bt819, bt856, zr36060, zr36067
 Inputs/outputs: Composite and S-video
 Norms: PAL (720x576 @ 25 fps), NTSC (720x480 @ 29.97 fps)
 Card number: 5
 Linux Media Labs LML33R10:
 * Zoran zr36067 PCI controller
 * Zoran zr36060 MJPEG codec
 * Philips saa7114 TV decoder
 * Analog Devices adv7170 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, saa7114, adv7170, zr36060, zr36067
 Inputs/outputs: Composite and S-video
 Norms: PAL (720x576 @ 25 fps), NTSC (720x480 @ 29.97 fps)
 Card number: 6
 Pinnacle/Miro DC10(new):
 * Zoran zr36057 PCI controller
 * Zoran zr36060 MJPEG codec
 * Philips saa7110a TV decoder
 * Analog Devices adv7176 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, saa7110, adv7175, zr36060, zr36067
 Inputs/outputs: Composite, S-video and Internal
 Norms: PAL, SECAM (768x576 @ 25 fps), NTSC (640x480 @ 29.97 fps)
 Card number: 1
 Pinnacle/Miro DC10+:
 * Zoran zr36067 PCI controller
 * Zoran zr36060 MJPEG codec
 * Philips saa7110a TV decoder
 * Analog Devices adv7176 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, sa7110, adv7175, zr36060, zr36067
 Inputs/outputs: Composite, S-video and Internal
 Norms: PAL, SECAM (768x576 @ 25 fps), NTSC (640x480 @ 29.97 fps)
 Card number: 2
 Pinnacle/Miro DC10(old): *
 * Zoran zr36057 PCI controller
 * Zoran zr36050 MJPEG codec
 * Zoran zr36016 Video Front End or Fuji md0211 Video Front End (clone?)
 * Micronas vpx3220a TV decoder
 * mse3000 TV encoder or Analog Devices adv7176 TV encoder *
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, vpx3220, mse3000/adv7175, zr36050, zr36016, zr36067
 Inputs/outputs: Composite, S-video and Internal
 Norms: PAL, SECAM (768x576 @ 25 fps), NTSC (640x480 @ 29.97 fps)
 Card number: 0
 Pinnacle/Miro DC30: *
 * Zoran zr36057 PCI controller
 * Zoran zr36050 MJPEG codec
 * Zoran zr36016 Video Front End
 * Micronas vpx3225d/vpx3220a/vpx3216b TV decoder
 * Analog Devices adv7176 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, vpx3220/vpx3224, adv7175, zr36050, zr36016, zr36067
 Inputs/outputs: Composite, S-video and Internal
 Norms: PAL, SECAM (768x576 @ 25 fps), NTSC (640x480 @ 29.97 fps)
 Card number: 3
 Pinnacle/Miro DC30+: *
 * Zoran zr36067 PCI controller
 * Zoran zr36050 MJPEG codec
 * Zoran zr36016 Video Front End
 * Micronas vpx3225d/vpx3220a/vpx3216b TV decoder
 * Analog Devices adv7176 TV encoder
 Drivers to use: videodev, i2c-core, i2c-algo-bit,
 		videocodec, vpx3220/vpx3224, adv7175, zr36050, zr36015, zr36067
 Inputs/outputs: Composite, S-video and Internal
 Norms: PAL, SECAM (768x576 @ 25 fps), NTSC (640x480 @ 29.97 fps)
 Card number: 4
 Note: No module for the mse3000 is available yet
 Note: No module for the vpx3224 is available yet
 Note: use encoder=X or decoder=X for non-default i2c chips (see i2c-id.h)
 ===========================
 1.1 What the TV decoder can do an what not
 The best know TV standards are NTSC/PAL/SECAM. but for decoding a frame that
 information is not enough. There are several formats of the TV standards.
 And not every TV decoder is able to handle every format. Also the every
 combination is supported by the driver. There are currently 11 different
 tv broadcast formats all aver the world.
 The CCIR defines parameters needed for broadcasting the signal.
 The CCIR has defined different standards: A,B,D,E,F,G,D,H,I,K,K1,L,M,N,...
-The CCIR says not much about about the colorsystem used !!!
+The CCIR says not much about the colorsystem used !!!
 And talking about a colorsystem says not to much about how it is broadcast.
 The CCIR standards A,E,F are not used any more.
 When you speak about NTSC, you usually mean the standard: CCIR - M using
 the NTSC colorsystem which is used in the USA, Japan, Mexico, Canada
 and a few others.
 When you talk about PAL, you usually mean: CCIR - B/G using the PAL
 colorsystem which is used in many Countries.
 When you talk about SECAM, you mean: CCIR - L using the SECAM Colorsystem
 which is used in France, and a few others.
 There the other version of SECAM, CCIR - D/K is used in Bulgaria, China,
 Slovakai, Hungary, Korea (Rep.), Poland, Rumania and a others.
 The CCIR - H uses the PAL colorsystem (sometimes SECAM) and is used in
 Egypt, Libya, Sri Lanka, Syrain Arab. Rep.
 The CCIR - I uses the PAL colorsystem, and is used in Great Britain, Hong Kong,
 Ireland, Nigeria, South Africa.
 The CCIR - N uses the PAL colorsystem and PAL frame size but the NTSC framerate,
 and is used in Argentinia, Uruguay, an a few others
 We do not talk about how the audio is broadcast !
 A rather good sites about the TV standards are:
 http://www.sony.jp/ServiceArea/Voltage_map/
 http://info.electronicwerkstatt.de/bereiche/fernsehtechnik/frequenzen_und_normen/Fernsehnormen/
 and http://www.cabl.com/restaurant/channel.html
 Other weird things around: NTSC 4.43 is a modificated NTSC, which is mainly
 used in PAL VCR's that are able to play back NTSC. PAL 60 seems to be the same
 as NTSC 4.43 . The Datasheets also talk about NTSC 44, It seems as if it would
 be the same as NTSC 4.43.
 NTSC Combs seems to be a decoder mode where the decoder uses a comb filter
 to split coma and luma instead of a Delay line.
 But I did not defiantly find out what NTSC Comb is.
 Philips saa7111 TV decoder
 was introduced in 1997, is used in the BUZ and
 can handle: PAL B/G/H/I, PAL N, PAL M, NTSC M, NTSC N, NTSC 4.43 and SECAM
 Philips saa7110a TV decoder
 was introduced in 1995, is used in the Pinnacle/Miro DC10(new), DC10+ and
 can handle: PAL B/G, NTSC M and SECAM
 Philips saa7114 TV decoder
 was introduced in 2000, is used in the LML33R10 and
 can handle: PAL B/G/D/H/I/N, PAL N, PAL M, NTSC M, NTSC 4.43 and SECAM
 Brooktree bt819 TV decoder
 was introduced in 1996, and is used in the LML33 and
 can handle: PAL B/D/G/H/I, NTSC M
 Micronas vpx3220a TV decoder
 was introduced in 1996, is used in the DC30 and DC30+ and
 can handle: PAL B/G/H/I, PAL N, PAL M, NTSC M, NTSC 44, PAL 60, SECAM,NTSC Comb
 Samsung ks0127 TV decoder
 is used in the AVS6EYES card and
 can handle: NTSC-M/N/44, PAL-M/N/B/G/H/I/D/K/L and SECAM
 ===========================
 1.2 What the TV encoder can do an what not
 The TV encoder are doing the "same" as the decoder, but in the oder direction.
 You feed them digital data and the generate a Composite or SVHS signal.
 For information about the colorsystems and TV norm take a look in the
 TV decoder section.
 Philips saa7185 TV Encoder
 was introduced in 1996, is used in the BUZ
 can generate: PAL B/G, NTSC M
 Brooktree bt856 TV Encoder
 was introduced in 1994, is used in the LML33
 can generate: PAL B/D/G/H/I/N, PAL M, NTSC M, PAL-N (Argentina)
 Analog Devices adv7170 TV Encoder
 was introduced in 2000, is used in the LML300R10
 can generate: PAL B/D/G/H/I/N, PAL M, NTSC M, PAL 60
 Analog Devices adv7175 TV Encoder
 was introduced in 1996, is used in the DC10, DC10+, DC10 old, DC30, DC30+
 can generate: PAL B/D/G/H/I/N, PAL M, NTSC M
 ITT mse3000 TV encoder
 was introduced in 1991, is used in the DC10 old
 can generate: PAL , NTSC , SECAM
 Conexant bt866 TV encoder
 is used in AVS6EYES, and
 can generate: NTSC/PAL, PALM, PALN
 The adv717x, should be able to produce PAL N. But you find nothing PAL N
 specific in the registers. Seem that you have to reuse a other standard
 to generate PAL N, maybe it would work if you use the PAL M settings.
 ==========================
 2. How do I get this damn thing to work
 Load zr36067.o. If it can't autodetect your card, use the card=X insmod
 option with X being the card number as given in the previous section.
 To have more than one card, use card=X1[,X2[,X3,[X4[..]]]]
 To automate this, add the following to your /etc/modprobe.conf:
 options zr36067 card=X1[,X2[,X3[,X4[..]]]]
 alias char-major-81-0 zr36067
 One thing to keep in mind is that this doesn't load zr36067.o itself yet. It
 just automates loading. If you start using xawtv, the device won't load on
 some systems, since you're trying to load modules as a user, which is not
 allowed ("permission denied"). A quick workaround is to add 'Load "v4l"' to
 XF86Config-4 when you use X by default, or to run 'v4l-conf -c <device>' in
 one of your startup scripts (normally rc.local) if you don't use X. Both
 make sure that the modules are loaded on startup, under the root account.
 ===========================
 3. What mainboard should I use (or why doesn't my card work)
 <insert lousy disclaimer here>. In short: good=SiS/Intel, bad=VIA.
 Experience tells us that people with a Buz, on average, have more problems
 than users with a DC10+/LML33. Also, it tells us that people owning a VIA-
 based mainboard (ktXXX, MVP3) have more problems than users with a mainboard
 based on a different chipset. Here's some notes from Andrew Stevens:
 --
 Here's my experience of using LML33 and Buz on various motherboards:
 VIA MVP3
 	Forget it. Pointless. Doesn't work.
 Intel 430FX (Pentium 200)
 	LML33 perfect, Buz tolerable (3 or 4 frames dropped per movie)
 Intel 440BX (early stepping)
 	LML33 tolerable. Buz starting to get annoying (6-10 frames/hour)
 Intel 440BX (late stepping)
 	Buz tolerable, LML3 almost perfect (occasional single frame drops)
 SiS735
 	LML33 perfect, Buz tolerable.
 VIA KT133(*)
 	LML33 starting to get annoying, Buz poor enough that I have up.
 Both 440BX boards were dual CPU versions.
 --
 Bernhard Praschinger later added:
 --
 AMD 751
 	Buz perfect-tolerable
 AMD 760
 	Buz perfect-tolerable
 --
 In general, people on the user mailinglist won't give you much of a chance
 if you have a VIA-based motherboard. They may be cheap, but sometimes, you'd
 rather want to spend some more money on better boards. In general, VIA
 mainboard's IDE/PCI performance will also suck badly compared to others.
 You'll noticed the DC10+/DC30+ aren't mentioned anywhere in the overview.
 Basically, you can assume that if the Buz works, the LML33 will work too. If
 the LML33 works, the DC10+/DC30+ will work too. They're most tolerant to
 different mainboard chipsets from all of the supported cards.
 If you experience timeouts during capture, buy a better mainboard or lower
 the quality/buffersize during capture (see 'Concerning buffer sizes, quality,
 output size etc.'). If it hangs, there's little we can do as of now. Check
 your IRQs and make sure the card has its own interrupts.
 ===========================
 4. Programming interface
 This driver conforms to video4linux and video4linux2, both can be used to
 use the driver. Since video4linux didn't provide adequate calls to fully
 use the cards' features, we've introduced several programming extensions,
 which are currently officially accepted in the 2.4.x branch of the kernel.
 These extensions are known as the v4l/mjpeg extensions. See zoran.h for
 details (structs/ioctls).
 Information - video4linux:
 http://roadrunner.swansea.linux.org.uk/v4lapi.shtml
 Documentation/video4linux/API.html
 /usr/include/linux/videodev.h
 Information - video4linux/mjpeg extensions:
 ./zoran.h
 (also see below)
 Information - video4linux2:
 http://www.thedirks.org/v4l2/
 /usr/include/linux/videodev2.h
 http://www.bytesex.org/v4l/
 More information on the video4linux/mjpeg extensions, by Serguei
 Miridonovi and Rainer Johanni:
 --
 The ioctls for that interface are as follows:
 BUZIOC_G_PARAMS
 BUZIOC_S_PARAMS
 Get and set the parameters of the buz. The user should always do a
 BUZIOC_G_PARAMS (with a struct buz_params) to obtain the default
 settings, change what he likes and then make a BUZIOC_S_PARAMS call.
 BUZIOC_REQBUFS
 Before being able to capture/playback, the user has to request
 the buffers he is wanting to use. Fill the structure
 zoran_requestbuffers with the size (recommended: 256*1024) and
 the number (recommended 32 up to 256). There are no such restrictions
 as for the Video for Linux buffers, you should LEAVE SUFFICIENT
 MEMORY for your system however, else strange things will happen ....
 On return, the zoran_requestbuffers structure contains number and
 size of the actually allocated buffers.
 You should use these numbers for doing a mmap of the buffers
 into the user space.
 The BUZIOC_REQBUFS ioctl also makes it happen, that the next mmap
 maps the MJPEG buffer instead of the V4L buffers.
 BUZIOC_QBUF_CAPT
 BUZIOC_QBUF_PLAY
 Queue a buffer for capture or playback. The first call also starts
 streaming capture. When streaming capture is going on, you may
 only queue further buffers or issue syncs until streaming
 capture is switched off again with a argument of -1 to
 a BUZIOC_QBUF_CAPT/BUZIOC_QBUF_PLAY ioctl.
 BUZIOC_SYNC
 Issue this ioctl when all buffers are queued. This ioctl will
 block until the first buffer becomes free for saving its
 data to disk (after BUZIOC_QBUF_CAPT) or for reuse (after BUZIOC_QBUF_PLAY).
 BUZIOC_G_STATUS
 Get the status of the input lines (video source connected/norm).
 For programming example, please, look at lavrec.c and lavplay.c code in
 lavtools-1.2p2 package (URL: http://www.cicese.mx/~mirsev/DC10plus/)
 and the 'examples' directory in the original Buz driver distribution.
 Additional notes for software developers:
   The driver returns maxwidth and maxheight parameters according to
   the current TV standard (norm). Therefore, the software which
   communicates with the driver and "asks" for these parameters should
   first set the correct norm. Well, it seems logically correct: TV
   standard is "more constant" for current country than geometry
   settings of a variety of TV capture cards which may work in ITU or
   square pixel format. Remember that users now can lock the norm to
   avoid any ambiguity.
 --
 Please note that lavplay/lavrec are also included in the MJPEG-tools
 (http://mjpeg.sf.net/).
 ===========================
 5. Applications
 Applications known to work with this driver:
 TV viewing:
 * xawtv
 * kwintv
 * probably any TV application that supports video4linux or video4linux2.
 MJPEG capture/playback:
 * mjpegtools/lavtools (or Linux Video Studio)
 * gstreamer
 * mplayer
 General raw capture:
 * xawtv
 * gstreamer
 * probably any application that supports video4linux or video4linux2
 Video editing:
 * Cinelerra
 * MainActor
 * mjpegtools (or Linux Video Studio)
 ===========================
 6. Concerning buffer sizes, quality, output size etc.
 The zr36060 can do 1:2 JPEG compression. This is really the theoretical
 maximum that the chipset can reach. The driver can, however, limit compression
 to a maximum (size) of 1:4. The reason for this is that some cards (e.g. Buz)
 can't handle 1:2 compression without stopping capture after only a few minutes.
 With 1:4, it'll mostly work. If you have a Buz, use 'low_bitrate=1' to go into
 1:4 max. compression mode.
 100% JPEG quality is thus 1:2 compression in practice. So for a full PAL frame
 (size 720x576). The JPEG fields are stored in YUY2 format, so the size of the
 fields are 720x288x16/2 bits/field (2 fields/frame) = 207360 bytes/field x 2 =
 414720 bytes/frame (add some more bytes for headers and DHT (huffman)/DQT
 (quantization) tables, and you'll get to something like 512kB per frame for
 1:2 compression. For 1:4 compression, you'd have frames of half this size.
 Some additional explanation by Martin Samuelsson, which also explains the
 importance of buffer sizes:
 --
 > Hmm, I do not think it is really that way. With the current (downloaded
 > at 18:00 Monday) driver I get that output sizes for 10 sec:
 > -q 50 -b 128 : 24.283.332 Bytes
 > -q 50 -b 256 : 48.442.368
 > -q 25 -b 128 : 24.655.992
 > -q 25 -b 256 : 25.859.820
 I woke up, and can't go to sleep again. I'll kill some time explaining why
 this doesn't look strange to me.
 Let's do some math using a width of 704 pixels. I'm not sure whether the Buz
 actually use that number or not, but that's not too important right now.
 704x288 pixels, one field, is 202752 pixels. Divided by 64 pixels per block;
 3168 blocks per field. Each pixel consist of two bytes; 128 bytes per block;
 1024 bits per block. 100% in the new driver mean 1:2 compression; the maximum
 output becomes 512 bits per block. Actually 510, but 512 is simpler to use
 for calculations.
 Let's say that we specify d1q50. We thus want 256 bits per block; times 3168
 becomes 811008 bits; 101376 bytes per field. We're talking raw bits and bytes
 here, so we don't need to do any fancy corrections for bits-per-pixel or such
 things. 101376 bytes per field.
 d1 video contains two fields per frame. Those sum up to 202752 bytes per
 frame, and one of those frames goes into each buffer.
 But wait a second! -b128 gives 128kB buffers! It's not possible to cram
 202752 bytes of JPEG data into 128kB!
 This is what the driver notice and automatically compensate for in your
 examples. Let's do some math using this information:
 128kB is 131072 bytes. In this buffer, we want to store two fields, which
 leaves 65536 bytes for each field. Using 3168 blocks per field, we get
 20.68686868... available bytes per block; 165 bits. We can't allow the
 request for 256 bits per block when there's only 165 bits available! The -q50
 option is silently overridden, and the -b128 option takes precedence, leaving
 us with the equivalence of -q32.
 This gives us a data rate of 165 bits per block, which, times 3168, sums up
 to 65340 bytes per field, out of the allowed 65536. The current driver has
 another level of rate limiting; it won't accept -q values that fill more than
 6/8 of the specified buffers. (I'm not sure why. "Playing it safe" seem to be
 a safe bet. Personally, I think I would have lowered requested-bits-per-block
 by one, or something like that.) We can't use 165 bits per block, but have to
 lower it again, to 6/8 of the available buffer space: We end up with 124 bits
 per block, the equivalence of -q24. With 128kB buffers, you can't use greater
 than -q24 at -d1. (And PAL, and 704 pixels width...)
 The third example is limited to -q24 through the same process. The second
 example, using very similar calculations, is limited to -q48. The only
 example that actually grab at the specified -q value is the last one, which
 is clearly visible, looking at the file size.
 --
 Conclusion: the quality of the resulting movie depends on buffer size, quality,
 whether or not you use 'low_bitrate=1' as insmod option for the zr36060.c
 module to do 1:4 instead of 1:2 compression, etc.
 If you experience timeouts, lowering the quality/buffersize or using
 'low_bitrate=1 as insmod option for zr36060.o might actually help, as is
 proven by the Buz.
 ===========================
 7. It hangs/crashes/fails/whatevers! Help!
 Make sure that the card has its own interrupts (see /proc/interrupts), check
 the output of dmesg at high verbosity (load zr36067.o with debug=2,
 load all other modules with debug=1). Check that your mainboard is favorable
 (see question 2) and if not, test the card in another computer. Also see the
 notes given in question 3 and try lowering quality/buffersize/capturesize
 if recording fails after a period of time.
 If all this doesn't help, give a clear description of the problem including
 detailed hardware information (memory+brand, mainboard+chipset+brand, which
 MJPEG card, processor, other PCI cards that might be of interest), give the
 system PnP information (/proc/interrupts, /proc/dma, /proc/devices), and give
 the kernel version, driver version, glibc version, gcc version and any other
 information that might possibly be of interest. Also provide the dmesg output
 at high verbosity. See 'Contacting' on how to contact the developers.
 ===========================
 8. Maintainers/Contacting
 The driver is currently maintained by Laurent Pinchart and Ronald Bultje
 (<laurent.pinchart@skynet.be> and <rbultje@ronald.bitfreak.net>). For bug
 reports or questions, please contact the mailinglist instead of the developers
 individually. For user questions (i.e. bug reports or how-to questions), send
 an email to <mjpeg-users@lists.sf.net>, for developers (i.e. if you want to
 help programming), send an email to <mjpeg-developer@lists.sf.net>. See
 http://www.sf.net/projects/mjpeg/ for subscription information.
 For bug reports, be sure to include all the information as described in
 the section 'It hangs/crashes/fails/whatevers! Help!'. Please make sure
 you're using the latest version (http://mjpeg.sf.net/driver-zoran/).
 Previous maintainers/developers of this driver include Serguei Miridonov
 <mirsev@cicese.mx>, Wolfgang Scherr <scherr@net4you.net>, Dave Perks
 <dperks@ibm.net> and Rainer Johanni <Rainer@Johanni.de>.
 ===========================
 9. License
 This driver is distributed under the terms of the General Public License.
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 See http://www.gnu.org/ for more information.
 Started Nov 1999 by Kanoj Sarcar <kanoj@sgi.com>
 The intent of this file is to have an uptodate, running commentary 
 from different people about NUMA specific code in the Linux vm.
 What is NUMA? It is an architecture where the memory access times
 for different regions of memory from a given processor varies
 according to the "distance" of the memory region from the processor.
 Each region of memory to which access times are the same from any 
 cpu, is called a node. On such architectures, it is beneficial if
 the kernel tries to minimize inter node communications. Schemes
 for this range from kernel text and read-only data replication
 across nodes, and trying to house all the data structures that
 key components of the kernel need on memory on that node.
 Currently, all the numa support is to provide efficient handling
 of widely discontiguous physical memory, so architectures which 
 are not NUMA but can have huge holes in the physical address space
 can use the same code. All this code is bracketed by CONFIG_DISCONTIGMEM.
 The initial port includes NUMAizing the bootmem allocator code by
 encapsulating all the pieces of information into a bootmem_data_t
 structure. Node specific calls have been added to the allocator. 
 In theory, any platform which uses the bootmem allocator should 
-be able to to put the bootmem and mem_map data structures anywhere
+be able to put the bootmem and mem_map data structures anywhere
 it deems best.
 Each node's page allocation data structures have also been encapsulated
 into a pg_data_t. The bootmem_data_t is just one part of this. To 
 make the code look uniform between NUMA and regular UMA platforms, 
 UMA platforms have a statically allocated pg_data_t too (contig_page_data).
 For the sake of uniformity, the function num_online_nodes() is also defined
 for all platforms. As we run benchmarks, we might decide to NUMAize 
 more variables like low_on_memory, nr_free_pages etc into the pg_data_t.
 The NUMA aware page allocation code currently tries to allocate pages 
 from different nodes in a round robin manner.  This will be changed to 
 do concentratic circle search, starting from current node, once the 
 NUMA port achieves more maturity. The call alloc_pages_node has been 
 added, so that drivers can make the call and not worry about whether 
 it is running on a NUMA or UMA platform.
1	The Linux Kernel Device Model	1	The Linux Kernel Device Model
2		2
3	Patrick Mochel <mochel@digitalimplant.org>	3	Patrick Mochel <mochel@digitalimplant.org>
4		4
5	Drafted 26 August 2002	5	Drafted 26 August 2002
6	Updated 31 January 2006	6	Updated 31 January 2006
7		7
8		8
9	Overview	9	Overview
10	~~~~~~~~	10	~~~~~~~~
11		11
12	The Linux Kernel Driver Model is a unification of all the disparate driver	12	The Linux Kernel Driver Model is a unification of all the disparate driver
13	models that were previously used in the kernel. It is intended to augment the	13	models that were previously used in the kernel. It is intended to augment the
14	bus-specific drivers for bridges and devices by consolidating a set of data	14	bus-specific drivers for bridges and devices by consolidating a set of data
15	and operations into globally accessible data structures.	15	and operations into globally accessible data structures.
16		16
17	Traditional driver models implemented some sort of tree-like structure	17	Traditional driver models implemented some sort of tree-like structure
18	(sometimes just a list) for the devices they control. There wasn't any	18	(sometimes just a list) for the devices they control. There wasn't any
19	uniformity across the different bus types.	19	uniformity across the different bus types.
20		20
21	The current driver model provides a common, uniform data model for describing	21	The current driver model provides a common, uniform data model for describing
22	a bus and the devices that can appear under the bus. The unified bus	22	a bus and the devices that can appear under the bus. The unified bus
23	model includes a set of common attributes which all busses carry, and a set	23	model includes a set of common attributes which all busses carry, and a set
24	of common callbacks, such as device discovery during bus probing, bus	24	of common callbacks, such as device discovery during bus probing, bus
25	shutdown, bus power management, etc.	25	shutdown, bus power management, etc.
26		26
27	The common device and bridge interface reflects the goals of the modern	27	The common device and bridge interface reflects the goals of the modern
28	computer: namely the ability to do seamless device "plug and play", power	28	computer: namely the ability to do seamless device "plug and play", power
29	management, and hot plug. In particular, the model dictated by Intel and	29	management, and hot plug. In particular, the model dictated by Intel and
30	Microsoft (namely ACPI) ensures that almost every device on almost any bus	30	Microsoft (namely ACPI) ensures that almost every device on almost any bus
31	on an x86-compatible system can work within this paradigm. Of course,	31	on an x86-compatible system can work within this paradigm. Of course,
32	not every bus is able to support all such operations, although most	32	not every bus is able to support all such operations, although most
33	buses support a most of those operations.	33	buses support a most of those operations.
34		34
35		35
36	Downstream Access	36	Downstream Access
37	~~~~~~~~~~~~~~~~~	37	~~~~~~~~~~~~~~~~~
38		38
39	Common data fields have been moved out of individual bus layers into a common	39	Common data fields have been moved out of individual bus layers into a common
40	data structure. These fields must still be accessed by the bus layers,	40	data structure. These fields must still be accessed by the bus layers,
41	and sometimes by the device-specific drivers.	41	and sometimes by the device-specific drivers.
42		42
43	Other bus layers are encouraged to do what has been done for the PCI layer.	43	Other bus layers are encouraged to do what has been done for the PCI layer.
44	struct pci_dev now looks like this:	44	struct pci_dev now looks like this:
45		45
46	struct pci_dev {	46	struct pci_dev {
47	...	47	...
48		48
49	struct device dev;	49	struct device dev;
50	};	50	};
51		51
52	Note first that it is statically allocated. This means only one allocation on	52	Note first that it is statically allocated. This means only one allocation on
53	device discovery. Note also that it is at the _end_ of struct pci_dev. This is	53	device discovery. Note also that it is at the _end_ of struct pci_dev. This is
54	to make people think about what they're doing when switching between the bus	54	to make people think about what they're doing when switching between the bus
55	driver and the global driver; and to prevent against mindless casts between	55	driver and the global driver; and to prevent against mindless casts between
56	the two.	56	the two.
57		57
58	The PCI bus layer freely accesses the fields of struct device. It knows about	58	The PCI bus layer freely accesses the fields of struct device. It knows about
59	the structure of struct pci_dev, and it should know the structure of struct	59	the structure of struct pci_dev, and it should know the structure of struct
60	device. Individual PCI device drivers that have been converted the the current	60	device. Individual PCI device drivers that have been converted to the current
61	driver model generally do not and should not touch the fields of struct device,	61	driver model generally do not and should not touch the fields of struct device,
62	unless there is a strong compelling reason to do so.	62	unless there is a strong compelling reason to do so.
63		63
64	This abstraction is prevention of unnecessary pain during transitional phases.	64	This abstraction is prevention of unnecessary pain during transitional phases.
65	If the name of the field changes or is removed, then every downstream driver	65	If the name of the field changes or is removed, then every downstream driver
66	will break. On the other hand, if only the bus layer (and not the device	66	will break. On the other hand, if only the bus layer (and not the device
67	layer) accesses struct device, it is only that layer that needs to change.	67	layer) accesses struct device, it is only that layer that needs to change.
68		68
69		69
70	User Interface	70	User Interface
71	~~~~~~~~~~~~~~	71	~~~~~~~~~~~~~~
72		72
73	By virtue of having a complete hierarchical view of all the devices in the	73	By virtue of having a complete hierarchical view of all the devices in the
74	system, exporting a complete hierarchical view to userspace becomes relatively	74	system, exporting a complete hierarchical view to userspace becomes relatively
75	easy. This has been accomplished by implementing a special purpose virtual	75	easy. This has been accomplished by implementing a special purpose virtual
76	file system named sysfs. It is hence possible for the user to mount the	76	file system named sysfs. It is hence possible for the user to mount the
77	whole sysfs filesystem anywhere in userspace.	77	whole sysfs filesystem anywhere in userspace.
78		78
79	This can be done permanently by providing the following entry into the	79	This can be done permanently by providing the following entry into the
80	/etc/fstab (under the provision that the mount point does exist, of course):	80	/etc/fstab (under the provision that the mount point does exist, of course):
81		81
82	none /sys sysfs defaults 0 0	82	none /sys sysfs defaults 0 0
83		83
84	Or by hand on the command line:	84	Or by hand on the command line:
85		85
86	# mount -t sysfs sysfs /sys	86	# mount -t sysfs sysfs /sys
87		87
88	Whenever a device is inserted into the tree, a directory is created for it.	88	Whenever a device is inserted into the tree, a directory is created for it.
89	This directory may be populated at each layer of discovery - the global layer,	89	This directory may be populated at each layer of discovery - the global layer,
90	the bus layer, or the device layer.	90	the bus layer, or the device layer.
91		91
92	The global layer currently creates two files - 'name' and 'power'. The	92	The global layer currently creates two files - 'name' and 'power'. The
93	former only reports the name of the device. The latter reports the	93	former only reports the name of the device. The latter reports the
94	current power state of the device. It will also be used to set the current	94	current power state of the device. It will also be used to set the current
95	power state.	95	power state.
96		96
97	The bus layer may also create files for the devices it finds while probing the	97	The bus layer may also create files for the devices it finds while probing the
98	bus. For example, the PCI layer currently creates 'irq' and 'resource' files	98	bus. For example, the PCI layer currently creates 'irq' and 'resource' files
99	for each PCI device.	99	for each PCI device.
100		100
101	A device-specific driver may also export files in its directory to expose	101	A device-specific driver may also export files in its directory to expose
102	device-specific data or tunable interfaces.	102	device-specific data or tunable interfaces.
103		103
104	More information about the sysfs directory layout can be found in	104	More information about the sysfs directory layout can be found in
105	the other documents in this directory and in the file	105	the other documents in this directory and in the file
106	Documentation/filesystems/sysfs.txt.	106	Documentation/filesystems/sysfs.txt.
107		107
108		108
1	Locking scheme used for directory operations is based on two	1	Locking scheme used for directory operations is based on two
2	kinds of locks - per-inode (->i_sem) and per-filesystem (->s_vfs_rename_sem).	2	kinds of locks - per-inode (->i_sem) and per-filesystem (->s_vfs_rename_sem).
3		3
4	For our purposes all operations fall in 5 classes:	4	For our purposes all operations fall in 5 classes:
5		5
6	1) read access. Locking rules: caller locks directory we are accessing.	6	1) read access. Locking rules: caller locks directory we are accessing.
7		7
8	2) object creation. Locking rules: same as above.	8	2) object creation. Locking rules: same as above.
9		9
10	3) object removal. Locking rules: caller locks parent, finds victim,	10	3) object removal. Locking rules: caller locks parent, finds victim,
11	locks victim and calls the method.	11	locks victim and calls the method.
12		12
13	4) rename() that is _not_ cross-directory. Locking rules: caller locks	13	4) rename() that is _not_ cross-directory. Locking rules: caller locks
14	the parent, finds source and target, if target already exists - locks it	14	the parent, finds source and target, if target already exists - locks it
15	and then calls the method.	15	and then calls the method.
16		16
17	5) link creation. Locking rules:	17	5) link creation. Locking rules:
18	* lock parent	18	* lock parent
19	* check that source is not a directory	19	* check that source is not a directory
20	* lock source	20	* lock source
21	* call the method.	21	* call the method.
22		22
23	6) cross-directory rename. The trickiest in the whole bunch. Locking	23	6) cross-directory rename. The trickiest in the whole bunch. Locking
24	rules:	24	rules:
25	* lock the filesystem	25	* lock the filesystem
26	* lock parents in "ancestors first" order.	26	* lock parents in "ancestors first" order.
27	* find source and target.	27	* find source and target.
28	* if old parent is equal to or is a descendent of target	28	* if old parent is equal to or is a descendent of target
29	fail with -ENOTEMPTY	29	fail with -ENOTEMPTY
30	* if new parent is equal to or is a descendent of source	30	* if new parent is equal to or is a descendent of source
31	fail with -ELOOP	31	fail with -ELOOP
32	* if target exists - lock it.	32	* if target exists - lock it.
33	* call the method.	33	* call the method.
34		34
35		35
36	The rules above obviously guarantee that all directories that are going to be	36	The rules above obviously guarantee that all directories that are going to be
37	read, modified or removed by method will be locked by caller.	37	read, modified or removed by method will be locked by caller.
38		38
39		39
40	If no directory is its own ancestor, the scheme above is deadlock-free.	40	If no directory is its own ancestor, the scheme above is deadlock-free.
41	Proof:	41	Proof:
42		42
43	First of all, at any moment we have a partial ordering of the	43	First of all, at any moment we have a partial ordering of the
44	objects - A < B iff A is an ancestor of B.	44	objects - A < B iff A is an ancestor of B.
45		45
46	That ordering can change. However, the following is true:	46	That ordering can change. However, the following is true:
47		47
48	(1) if object removal or non-cross-directory rename holds lock on A and	48	(1) if object removal or non-cross-directory rename holds lock on A and
49	attempts to acquire lock on B, A will remain the parent of B until we	49	attempts to acquire lock on B, A will remain the parent of B until we
50	acquire the lock on B. (Proof: only cross-directory rename can change	50	acquire the lock on B. (Proof: only cross-directory rename can change
51	the parent of object and it would have to lock the parent).	51	the parent of object and it would have to lock the parent).
52		52
53	(2) if cross-directory rename holds the lock on filesystem, order will not	53	(2) if cross-directory rename holds the lock on filesystem, order will not
54	change until rename acquires all locks. (Proof: other cross-directory	54	change until rename acquires all locks. (Proof: other cross-directory
55	renames will be blocked on filesystem lock and we don't start changing	55	renames will be blocked on filesystem lock and we don't start changing
56	the order until we had acquired all locks).	56	the order until we had acquired all locks).
57		57
58	(3) any operation holds at most one lock on non-directory object and	58	(3) any operation holds at most one lock on non-directory object and
59	that lock is acquired after all other locks. (Proof: see descriptions	59	that lock is acquired after all other locks. (Proof: see descriptions
60	of operations).	60	of operations).
61		61
62	Now consider the minimal deadlock. Each process is blocked on	62	Now consider the minimal deadlock. Each process is blocked on
63	attempt to acquire some lock and already holds at least one lock. Let's	63	attempt to acquire some lock and already holds at least one lock. Let's
64	consider the set of contended locks. First of all, filesystem lock is	64	consider the set of contended locks. First of all, filesystem lock is
65	not contended, since any process blocked on it is not holding any locks.	65	not contended, since any process blocked on it is not holding any locks.
66	Thus all processes are blocked on ->i_sem.	66	Thus all processes are blocked on ->i_sem.
67		67
68	Non-directory objects are not contended due to (3). Thus link	68	Non-directory objects are not contended due to (3). Thus link
69	creation can't be a part of deadlock - it can't be blocked on source	69	creation can't be a part of deadlock - it can't be blocked on source
70	and it means that it doesn't hold any locks.	70	and it means that it doesn't hold any locks.
71		71
72	Any contended object is either held by cross-directory rename or	72	Any contended object is either held by cross-directory rename or
73	has a child that is also contended. Indeed, suppose that it is held by	73	has a child that is also contended. Indeed, suppose that it is held by
74	operation other than cross-directory rename. Then the lock this operation	74	operation other than cross-directory rename. Then the lock this operation
75	is blocked on belongs to child of that object due to (1).	75	is blocked on belongs to child of that object due to (1).
76		76
77	It means that one of the operations is cross-directory rename.	77	It means that one of the operations is cross-directory rename.
78	Otherwise the set of contended objects would be infinite - each of them	78	Otherwise the set of contended objects would be infinite - each of them
79	would have a contended child and we had assumed that no object is its	79	would have a contended child and we had assumed that no object is its
80	own descendent. Moreover, there is exactly one cross-directory rename	80	own descendent. Moreover, there is exactly one cross-directory rename
81	(see above).	81	(see above).
82		82
83	Consider the object blocking the cross-directory rename. One	83	Consider the object blocking the cross-directory rename. One
84	of its descendents is locked by cross-directory rename (otherwise we	84	of its descendents is locked by cross-directory rename (otherwise we
85	would again have an infinite set of of contended objects). But that	85	would again have an infinite set of contended objects). But that
86	means that cross-directory rename is taking locks out of order. Due	86	means that cross-directory rename is taking locks out of order. Due
87	to (2) the order hadn't changed since we had acquired filesystem lock.	87	to (2) the order hadn't changed since we had acquired filesystem lock.
88	But locking rules for cross-directory rename guarantee that we do not	88	But locking rules for cross-directory rename guarantee that we do not
89	try to acquire lock on descendent before the lock on ancestor.	89	try to acquire lock on descendent before the lock on ancestor.
90	Contradiction. I.e. deadlock is impossible. Q.E.D.	90	Contradiction. I.e. deadlock is impossible. Q.E.D.
91		91
92		92
93	These operations are guaranteed to avoid loop creation. Indeed,	93	These operations are guaranteed to avoid loop creation. Indeed,
94	the only operation that could introduce loops is cross-directory rename.	94	the only operation that could introduce loops is cross-directory rename.
95	Since the only new (parent, child) pair added by rename() is (new parent,	95	Since the only new (parent, child) pair added by rename() is (new parent,
96	source), such loop would have to contain these objects and the rest of it	96	source), such loop would have to contain these objects and the rest of it
97	would have to exist before rename(). I.e. at the moment of loop creation	97	would have to exist before rename(). I.e. at the moment of loop creation
98	rename() responsible for that would be holding filesystem lock and new parent	98	rename() responsible for that would be holding filesystem lock and new parent
99	would have to be equal to or a descendent of source. But that means that	99	would have to be equal to or a descendent of source. But that means that
100	new parent had been equal to or a descendent of source since the moment when	100	new parent had been equal to or a descendent of source since the moment when
101	we had acquired filesystem lock and rename() would fail with -ELOOP in that	101	we had acquired filesystem lock and rename() would fail with -ELOOP in that
102	case.	102	case.
103		103
104	While this locking scheme works for arbitrary DAGs, it relies on	104	While this locking scheme works for arbitrary DAGs, it relies on
105	ability to check that directory is a descendent of another object. Current	105	ability to check that directory is a descendent of another object. Current
106	implementation assumes that directory graph is a tree. This assumption is	106	implementation assumes that directory graph is a tree. This assumption is
107	also preserved by all operations (cross-directory rename on a tree that would	107	also preserved by all operations (cross-directory rename on a tree that would
108	not introduce a cycle will leave it a tree and link() fails for directories).	108	not introduce a cycle will leave it a tree and link() fails for directories).
109		109
110	Notice that "directory" in the above == "anything that might have	110	Notice that "directory" in the above == "anything that might have
111	children", so if we are going to introduce hybrid objects we will need	111	children", so if we are going to introduce hybrid objects we will need
112	either to make sure that link(2) doesn't work for them or to make changes	112	either to make sure that link(2) doesn't work for them or to make changes
113	in is_subdir() that would make it work even in presence of such beasts.	113	in is_subdir() that would make it work even in presence of such beasts.
114		114
1	File management in the Linux kernel	1	File management in the Linux kernel
2	-----------------------------------	2	-----------------------------------
3		3
4	This document describes how locking for files (struct file)	4	This document describes how locking for files (struct file)
5	and file descriptor table (struct files) works.	5	and file descriptor table (struct files) works.
6		6
7	Up until 2.6.12, the file descriptor table has been protected	7	Up until 2.6.12, the file descriptor table has been protected
8	with a lock (files->file_lock) and reference count (files->count).	8	with a lock (files->file_lock) and reference count (files->count).
9	->file_lock protected accesses to all the file related fields	9	->file_lock protected accesses to all the file related fields
10	of the table. ->count was used for sharing the file descriptor	10	of the table. ->count was used for sharing the file descriptor
11	table between tasks cloned with CLONE_FILES flag. Typically	11	table between tasks cloned with CLONE_FILES flag. Typically
12	this would be the case for posix threads. As with the common	12	this would be the case for posix threads. As with the common
13	refcounting model in the kernel, the last task doing	13	refcounting model in the kernel, the last task doing
14	a put_files_struct() frees the file descriptor (fd) table.	14	a put_files_struct() frees the file descriptor (fd) table.
15	The files (struct file) themselves are protected using	15	The files (struct file) themselves are protected using
16	reference count (->f_count).	16	reference count (->f_count).
17		17
18	In the new lock-free model of file descriptor management,	18	In the new lock-free model of file descriptor management,
19	the reference counting is similar, but the locking is	19	the reference counting is similar, but the locking is
20	based on RCU. The file descriptor table contains multiple	20	based on RCU. The file descriptor table contains multiple
21	elements - the fd sets (open_fds and close_on_exec, the	21	elements - the fd sets (open_fds and close_on_exec, the
22	array of file pointers, the sizes of the sets and the array	22	array of file pointers, the sizes of the sets and the array
23	etc.). In order for the updates to appear atomic to	23	etc.). In order for the updates to appear atomic to
24	a lock-free reader, all the elements of the file descriptor	24	a lock-free reader, all the elements of the file descriptor
25	table are in a separate structure - struct fdtable.	25	table are in a separate structure - struct fdtable.
26	files_struct contains a pointer to struct fdtable through	26	files_struct contains a pointer to struct fdtable through
27	which the actual fd table is accessed. Initially the	27	which the actual fd table is accessed. Initially the
28	fdtable is embedded in files_struct itself. On a subsequent	28	fdtable is embedded in files_struct itself. On a subsequent
29	expansion of fdtable, a new fdtable structure is allocated	29	expansion of fdtable, a new fdtable structure is allocated
30	and files->fdtab points to the new structure. The fdtable	30	and files->fdtab points to the new structure. The fdtable
31	structure is freed with RCU and lock-free readers either	31	structure is freed with RCU and lock-free readers either
32	see the old fdtable or the new fdtable making the update	32	see the old fdtable or the new fdtable making the update
33	appear atomic. Here are the locking rules for	33	appear atomic. Here are the locking rules for
34	the fdtable structure -	34	the fdtable structure -
35		35
36	1. All references to the fdtable must be done through	36	1. All references to the fdtable must be done through
37	the files_fdtable() macro :	37	the files_fdtable() macro :
38		38
39	struct fdtable *fdt;	39	struct fdtable *fdt;
40		40
41	rcu_read_lock();	41	rcu_read_lock();
42		42
43	fdt = files_fdtable(files);	43	fdt = files_fdtable(files);
44	....	44	....
45	if (n <= fdt->max_fds)	45	if (n <= fdt->max_fds)
46	....	46	....
47	...	47	...
48	rcu_read_unlock();	48	rcu_read_unlock();
49		49
50	files_fdtable() uses rcu_dereference() macro which takes care of	50	files_fdtable() uses rcu_dereference() macro which takes care of
51	the memory barrier requirements for lock-free dereference.	51	the memory barrier requirements for lock-free dereference.
52	The fdtable pointer must be read within the read-side	52	The fdtable pointer must be read within the read-side
53	critical section.	53	critical section.
54		54
55	2. Reading of the fdtable as described above must be protected	55	2. Reading of the fdtable as described above must be protected
56	by rcu_read_lock()/rcu_read_unlock().	56	by rcu_read_lock()/rcu_read_unlock().
57		57
58	3. For any update to the the fd table, files->file_lock must	58	3. For any update to the fd table, files->file_lock must
59	be held.	59	be held.
60		60
61	4. To look up the file structure given an fd, a reader	61	4. To look up the file structure given an fd, a reader
62	must use either fcheck() or fcheck_files() APIs. These	62	must use either fcheck() or fcheck_files() APIs. These
63	take care of barrier requirements due to lock-free lookup.	63	take care of barrier requirements due to lock-free lookup.
64	An example :	64	An example :
65		65
66	struct file *file;	66	struct file *file;
67		67
68	rcu_read_lock();	68	rcu_read_lock();
69	file = fcheck(fd);	69	file = fcheck(fd);
70	if (file) {	70	if (file) {
71	...	71	...
72	}	72	}
73	....	73	....
74	rcu_read_unlock();	74	rcu_read_unlock();
75		75
76	5. Handling of the file structures is special. Since the look-up	76	5. Handling of the file structures is special. Since the look-up
77	of the fd (fget()/fget_light()) are lock-free, it is possible	77	of the fd (fget()/fget_light()) are lock-free, it is possible
78	that look-up may race with the last put() operation on the	78	that look-up may race with the last put() operation on the
79	file structure. This is avoided using the rcuref APIs	79	file structure. This is avoided using the rcuref APIs
80	on ->f_count :	80	on ->f_count :
81		81
82	rcu_read_lock();	82	rcu_read_lock();
83	file = fcheck_files(files, fd);	83	file = fcheck_files(files, fd);
84	if (file) {	84	if (file) {
85	if (rcuref_inc_lf(&file->f_count))	85	if (rcuref_inc_lf(&file->f_count))
86	*fput_needed = 1;	86	*fput_needed = 1;
87	else	87	else
88	/* Didn't get the reference, someone's freed */	88	/* Didn't get the reference, someone's freed */
89	file = NULL;	89	file = NULL;
90	}	90	}
91	rcu_read_unlock();	91	rcu_read_unlock();
92	....	92	....
93	return file;	93	return file;
94		94
95	rcuref_inc_lf() detects if refcounts is already zero or	95	rcuref_inc_lf() detects if refcounts is already zero or
96	goes to zero during increment. If it does, we fail	96	goes to zero during increment. If it does, we fail
97	fget()/fget_light().	97	fget()/fget_light().
98		98
99	6. Since both fdtable and file structures can be looked up	99	6. Since both fdtable and file structures can be looked up
100	lock-free, they must be installed using rcu_assign_pointer()	100	lock-free, they must be installed using rcu_assign_pointer()
101	API. If they are looked up lock-free, rcu_dereference()	101	API. If they are looked up lock-free, rcu_dereference()
102	must be used. However it is advisable to use files_fdtable()	102	must be used. However it is advisable to use files_fdtable()
103	and fcheck()/fcheck_files() which take care of these issues.	103	and fcheck()/fcheck_files() which take care of these issues.
104		104
105	7. While updating, the fdtable pointer must be looked up while	105	7. While updating, the fdtable pointer must be looked up while
106	holding files->file_lock. If ->file_lock is dropped, then	106	holding files->file_lock. If ->file_lock is dropped, then
107	another thread expand the files thereby creating a new	107	another thread expand the files thereby creating a new
108	fdtable and making the earlier fdtable pointer stale.	108	fdtable and making the earlier fdtable pointer stale.
109	For example :	109	For example :
110		110
111	spin_lock(&files->file_lock);	111	spin_lock(&files->file_lock);
112	fd = locate_fd(files, file, start);	112	fd = locate_fd(files, file, start);
113	if (fd >= 0) {	113	if (fd >= 0) {
114	/* locate_fd() may have expanded fdtable, load the ptr */	114	/* locate_fd() may have expanded fdtable, load the ptr */
115	fdt = files_fdtable(files);	115	fdt = files_fdtable(files);
116	FD_SET(fd, fdt->open_fds);	116	FD_SET(fd, fdt->open_fds);
117	FD_CLR(fd, fdt->close_on_exec);	117	FD_CLR(fd, fdt->close_on_exec);
118	spin_unlock(&files->file_lock);	118	spin_unlock(&files->file_lock);
119	.....	119	.....
120		120
121	Since locate_fd() can drop ->file_lock (and reacquire ->file_lock),	121	Since locate_fd() can drop ->file_lock (and reacquire ->file_lock),
122	the fdtable pointer (fdt) must be loaded after locate_fd().	122	the fdtable pointer (fdt) must be loaded after locate_fd().
123		123
124		124
1	Tmpfs is a file system which keeps all files in virtual memory.	1	Tmpfs is a file system which keeps all files in virtual memory.
2		2
3		3
4	Everything in tmpfs is temporary in the sense that no files will be	4	Everything in tmpfs is temporary in the sense that no files will be
5	created on your hard drive. If you unmount a tmpfs instance,	5	created on your hard drive. If you unmount a tmpfs instance,
6	everything stored therein is lost.	6	everything stored therein is lost.
7		7
8	tmpfs puts everything into the kernel internal caches and grows and	8	tmpfs puts everything into the kernel internal caches and grows and
9	shrinks to accommodate the files it contains and is able to swap	9	shrinks to accommodate the files it contains and is able to swap
10	unneeded pages out to swap space. It has maximum size limits which can	10	unneeded pages out to swap space. It has maximum size limits which can
11	be adjusted on the fly via 'mount -o remount ...'	11	be adjusted on the fly via 'mount -o remount ...'
12		12
13	If you compare it to ramfs (which was the template to create tmpfs)	13	If you compare it to ramfs (which was the template to create tmpfs)
14	you gain swapping and limit checking. Another similar thing is the RAM	14	you gain swapping and limit checking. Another similar thing is the RAM
15	disk (/dev/ram*), which simulates a fixed size hard disk in physical	15	disk (/dev/ram*), which simulates a fixed size hard disk in physical
16	RAM, where you have to create an ordinary filesystem on top. Ramdisks	16	RAM, where you have to create an ordinary filesystem on top. Ramdisks
17	cannot swap and you do not have the possibility to resize them.	17	cannot swap and you do not have the possibility to resize them.
18		18
19	Since tmpfs lives completely in the page cache and on swap, all tmpfs	19	Since tmpfs lives completely in the page cache and on swap, all tmpfs
20	pages currently in memory will show up as cached. It will not show up	20	pages currently in memory will show up as cached. It will not show up
21	as shared or something like that. Further on you can check the actual	21	as shared or something like that. Further on you can check the actual
22	RAM+swap use of a tmpfs instance with df(1) and du(1).	22	RAM+swap use of a tmpfs instance with df(1) and du(1).
23		23
24		24
25	tmpfs has the following uses:	25	tmpfs has the following uses:
26		26
27	1) There is always a kernel internal mount which you will not see at	27	1) There is always a kernel internal mount which you will not see at
28	all. This is used for shared anonymous mappings and SYSV shared	28	all. This is used for shared anonymous mappings and SYSV shared
29	memory.	29	memory.
30		30
31	This mount does not depend on CONFIG_TMPFS. If CONFIG_TMPFS is not	31	This mount does not depend on CONFIG_TMPFS. If CONFIG_TMPFS is not
32	set, the user visible part of tmpfs is not build. But the internal	32	set, the user visible part of tmpfs is not build. But the internal
33	mechanisms are always present.	33	mechanisms are always present.
34		34
35	2) glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for	35	2) glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for
36	POSIX shared memory (shm_open, shm_unlink). Adding the following	36	POSIX shared memory (shm_open, shm_unlink). Adding the following
37	line to /etc/fstab should take care of this:	37	line to /etc/fstab should take care of this:
38		38
39	tmpfs /dev/shm tmpfs defaults 0 0	39	tmpfs /dev/shm tmpfs defaults 0 0
40		40
41	Remember to create the directory that you intend to mount tmpfs on	41	Remember to create the directory that you intend to mount tmpfs on
42	if necessary.	42	if necessary.
43		43
44	This mount is _not_ needed for SYSV shared memory. The internal	44	This mount is _not_ needed for SYSV shared memory. The internal
45	mount is used for that. (In the 2.3 kernel versions it was	45	mount is used for that. (In the 2.3 kernel versions it was
46	necessary to mount the predecessor of tmpfs (shm fs) to use SYSV	46	necessary to mount the predecessor of tmpfs (shm fs) to use SYSV
47	shared memory)	47	shared memory)
48		48
49	3) Some people (including me) find it very convenient to mount it	49	3) Some people (including me) find it very convenient to mount it
50	e.g. on /tmp and /var/tmp and have a big swap partition. And now	50	e.g. on /tmp and /var/tmp and have a big swap partition. And now
51	loop mounts of tmpfs files do work, so mkinitrd shipped by most	51	loop mounts of tmpfs files do work, so mkinitrd shipped by most
52	distributions should succeed with a tmpfs /tmp.	52	distributions should succeed with a tmpfs /tmp.
53		53
54	4) And probably a lot more I do not know about :-)	54	4) And probably a lot more I do not know about :-)
55		55
56		56
57	tmpfs has three mount options for sizing:	57	tmpfs has three mount options for sizing:
58		58
59	size: The limit of allocated bytes for this tmpfs instance. The	59	size: The limit of allocated bytes for this tmpfs instance. The
60	default is half of your physical RAM without swap. If you	60	default is half of your physical RAM without swap. If you
61	oversize your tmpfs instances the machine will deadlock	61	oversize your tmpfs instances the machine will deadlock
62	since the OOM handler will not be able to free that memory.	62	since the OOM handler will not be able to free that memory.
63	nr_blocks: The same as size, but in blocks of PAGE_CACHE_SIZE.	63	nr_blocks: The same as size, but in blocks of PAGE_CACHE_SIZE.
64	nr_inodes: The maximum number of inodes for this instance. The default	64	nr_inodes: The maximum number of inodes for this instance. The default
65	is half of the number of your physical RAM pages, or (on a	65	is half of the number of your physical RAM pages, or (on a
66	a machine with highmem) the number of lowmem RAM pages,	66	machine with highmem) the number of lowmem RAM pages,
67	whichever is the lower.	67	whichever is the lower.
68		68
69	These parameters accept a suffix k, m or g for kilo, mega and giga and	69	These parameters accept a suffix k, m or g for kilo, mega and giga and
70	can be changed on remount. The size parameter also accepts a suffix %	70	can be changed on remount. The size parameter also accepts a suffix %
71	to limit this tmpfs instance to that percentage of your physical RAM:	71	to limit this tmpfs instance to that percentage of your physical RAM:
72	the default, when neither size nor nr_blocks is specified, is size=50%	72	the default, when neither size nor nr_blocks is specified, is size=50%
73		73
74	If nr_blocks=0 (or size=0), blocks will not be limited in that instance;	74	If nr_blocks=0 (or size=0), blocks will not be limited in that instance;
75	if nr_inodes=0, inodes will not be limited. It is generally unwise to	75	if nr_inodes=0, inodes will not be limited. It is generally unwise to
76	mount with such options, since it allows any user with write access to	76	mount with such options, since it allows any user with write access to
77	use up all the memory on the machine; but enhances the scalability of	77	use up all the memory on the machine; but enhances the scalability of
78	that instance in a system with many cpus making intensive use of it.	78	that instance in a system with many cpus making intensive use of it.
79		79
80		80
81	tmpfs has a mount option to set the NUMA memory allocation policy for	81	tmpfs has a mount option to set the NUMA memory allocation policy for
82	all files in that instance (if CONFIG_NUMA is enabled) - which can be	82	all files in that instance (if CONFIG_NUMA is enabled) - which can be
83	adjusted on the fly via 'mount -o remount ...'	83	adjusted on the fly via 'mount -o remount ...'
84		84
85	mpol=default prefers to allocate memory from the local node	85	mpol=default prefers to allocate memory from the local node
86	mpol=prefer:Node prefers to allocate memory from the given Node	86	mpol=prefer:Node prefers to allocate memory from the given Node
87	mpol=bind:NodeList allocates memory only from nodes in NodeList	87	mpol=bind:NodeList allocates memory only from nodes in NodeList
88	mpol=interleave prefers to allocate from each node in turn	88	mpol=interleave prefers to allocate from each node in turn
89	mpol=interleave:NodeList allocates from each node of NodeList in turn	89	mpol=interleave:NodeList allocates from each node of NodeList in turn
90		90
91	NodeList format is a comma-separated list of decimal numbers and ranges,	91	NodeList format is a comma-separated list of decimal numbers and ranges,
92	a range being two hyphen-separated decimal numbers, the smallest and	92	a range being two hyphen-separated decimal numbers, the smallest and
93	largest node numbers in the range. For example, mpol=bind:0-3,5,7,9-15	93	largest node numbers in the range. For example, mpol=bind:0-3,5,7,9-15
94		94
95	Note that trying to mount a tmpfs with an mpol option will fail if the	95	Note that trying to mount a tmpfs with an mpol option will fail if the
96	running kernel does not support NUMA; and will fail if its nodelist	96	running kernel does not support NUMA; and will fail if its nodelist
97	specifies a node >= MAX_NUMNODES. If your system relies on that tmpfs	97	specifies a node >= MAX_NUMNODES. If your system relies on that tmpfs
98	being mounted, but from time to time runs a kernel built without NUMA	98	being mounted, but from time to time runs a kernel built without NUMA
99	capability (perhaps a safe recovery kernel), or configured to support	99	capability (perhaps a safe recovery kernel), or configured to support
100	fewer nodes, then it is advisable to omit the mpol option from automatic	100	fewer nodes, then it is advisable to omit the mpol option from automatic
101	mount options. It can be added later, when the tmpfs is already mounted	101	mount options. It can be added later, when the tmpfs is already mounted
102	on MountPoint, by 'mount -o remount,mpol=Policy:NodeList MountPoint'.	102	on MountPoint, by 'mount -o remount,mpol=Policy:NodeList MountPoint'.
103		103
104		104
105	To specify the initial root directory you can use the following mount	105	To specify the initial root directory you can use the following mount
106	options:	106	options:
107		107
108	mode: The permissions as an octal number	108	mode: The permissions as an octal number
109	uid: The user id	109	uid: The user id
110	gid: The group id	110	gid: The group id
111		111
112	These options do not have any effect on remount. You can change these	112	These options do not have any effect on remount. You can change these
113	parameters with chmod(1), chown(1) and chgrp(1) on a mounted filesystem.	113	parameters with chmod(1), chown(1) and chgrp(1) on a mounted filesystem.
114		114
115		115
116	So 'mount -t tmpfs -o size=10G,nr_inodes=10k,mode=700 tmpfs /mytmpfs'	116	So 'mount -t tmpfs -o size=10G,nr_inodes=10k,mode=700 tmpfs /mytmpfs'
117	will give you tmpfs instance on /mytmpfs which can allocate 10GB	117	will give you tmpfs instance on /mytmpfs which can allocate 10GB
118	RAM/SWAP in 10240 inodes and it is only accessible by root.	118	RAM/SWAP in 10240 inodes and it is only accessible by root.
119		119
120		120
121	Author:	121	Author:
122	Christoph Rohland <cr@sap.com>, 1.12.01	122	Christoph Rohland <cr@sap.com>, 1.12.01
123	Updated:	123	Updated:
124	Hugh Dickins <hugh@veritas.com>, 19 February 2006	124	Hugh Dickins <hugh@veritas.com>, 19 February 2006
125		125
1	USING VFAT	1	USING VFAT
2	----------------------------------------------------------------------	2	----------------------------------------------------------------------
3	To use the vfat filesystem, use the filesystem type 'vfat'. i.e.	3	To use the vfat filesystem, use the filesystem type 'vfat'. i.e.
4	mount -t vfat /dev/fd0 /mnt	4	mount -t vfat /dev/fd0 /mnt
5		5
6	No special partition formatter is required. mkdosfs will work fine	6	No special partition formatter is required. mkdosfs will work fine
7	if you want to format from within Linux.	7	if you want to format from within Linux.
8		8
9	VFAT MOUNT OPTIONS	9	VFAT MOUNT OPTIONS
10	----------------------------------------------------------------------	10	----------------------------------------------------------------------
11	umask=### -- The permission mask (for files and directories, see umask(1)).	11	umask=### -- The permission mask (for files and directories, see umask(1)).
12	The default is the umask of current process.	12	The default is the umask of current process.
13		13
14	dmask=### -- The permission mask for the directory.	14	dmask=### -- The permission mask for the directory.
15	The default is the umask of current process.	15	The default is the umask of current process.
16		16
17	fmask=### -- The permission mask for files.	17	fmask=### -- The permission mask for files.
18	The default is the umask of current process.	18	The default is the umask of current process.
19		19
20	codepage=### -- Sets the codepage number for converting to shortname	20	codepage=### -- Sets the codepage number for converting to shortname
21	characters on FAT filesystem.	21	characters on FAT filesystem.
22	By default, FAT_DEFAULT_CODEPAGE setting is used.	22	By default, FAT_DEFAULT_CODEPAGE setting is used.
23		23
24	iocharset=name -- Character set to use for converting between the	24	iocharset=name -- Character set to use for converting between the
25	encoding is used for user visible filename and 16 bit	25	encoding is used for user visible filename and 16 bit
26	Unicode characters. Long filenames are stored on disk	26	Unicode characters. Long filenames are stored on disk
27	in Unicode format, but Unix for the most part doesn't	27	in Unicode format, but Unix for the most part doesn't
28	know how to deal with Unicode.	28	know how to deal with Unicode.
29	By default, FAT_DEFAULT_IOCHARSET setting is used.	29	By default, FAT_DEFAULT_IOCHARSET setting is used.
30		30
31	There is also an option of doing UTF-8 translations	31	There is also an option of doing UTF-8 translations
32	with the utf8 option.	32	with the utf8 option.
33		33
34	NOTE: "iocharset=utf8" is not recommended. If unsure,	34	NOTE: "iocharset=utf8" is not recommended. If unsure,
35	you should consider the following option instead.	35	you should consider the following option instead.
36		36
37	utf8=<bool> -- UTF-8 is the filesystem safe version of Unicode that	37	utf8=<bool> -- UTF-8 is the filesystem safe version of Unicode that
38	is used by the console. It can be be enabled for the	38	is used by the console. It can be enabled for the
39	filesystem with this option. If 'uni_xlate' gets set,	39	filesystem with this option. If 'uni_xlate' gets set,
40	UTF-8 gets disabled.	40	UTF-8 gets disabled.
41		41
42	uni_xlate=<bool> -- Translate unhandled Unicode characters to special	42	uni_xlate=<bool> -- Translate unhandled Unicode characters to special
43	escaped sequences. This would let you backup and	43	escaped sequences. This would let you backup and
44	restore filenames that are created with any Unicode	44	restore filenames that are created with any Unicode
45	characters. Until Linux supports Unicode for real,	45	characters. Until Linux supports Unicode for real,
46	this gives you an alternative. Without this option,	46	this gives you an alternative. Without this option,
47	a '?' is used when no translation is possible. The	47	a '?' is used when no translation is possible. The
48	escape character is ':' because it is otherwise	48	escape character is ':' because it is otherwise
49	illegal on the vfat filesystem. The escape sequence	49	illegal on the vfat filesystem. The escape sequence
50	that gets used is ':' and the four digits of hexadecimal	50	that gets used is ':' and the four digits of hexadecimal
51	unicode.	51	unicode.
52		52
53	nonumtail=<bool> -- When creating 8.3 aliases, normally the alias will	53	nonumtail=<bool> -- When creating 8.3 aliases, normally the alias will
54	end in '~1' or tilde followed by some number. If this	54	end in '~1' or tilde followed by some number. If this
55	option is set, then if the filename is	55	option is set, then if the filename is
56	"longfilename.txt" and "longfile.txt" does not	56	"longfilename.txt" and "longfile.txt" does not
57	currently exist in the directory, 'longfile.txt' will	57	currently exist in the directory, 'longfile.txt' will
58	be the short alias instead of 'longfi~1.txt'.	58	be the short alias instead of 'longfi~1.txt'.
59		59
60	quiet -- Stops printing certain warning messages.	60	quiet -- Stops printing certain warning messages.
61		61
62	check=s\|r\|n -- Case sensitivity checking setting.	62	check=s\|r\|n -- Case sensitivity checking setting.
63	s: strict, case sensitive	63	s: strict, case sensitive
64	r: relaxed, case insensitive	64	r: relaxed, case insensitive
65	n: normal, default setting, currently case insensitive	65	n: normal, default setting, currently case insensitive
66		66
67	shortname=lower\|win95\|winnt\|mixed	67	shortname=lower\|win95\|winnt\|mixed
68	-- Shortname display/create setting.	68	-- Shortname display/create setting.
69	lower: convert to lowercase for display,	69	lower: convert to lowercase for display,
70	emulate the Windows 95 rule for create.	70	emulate the Windows 95 rule for create.
71	win95: emulate the Windows 95 rule for display/create.	71	win95: emulate the Windows 95 rule for display/create.
72	winnt: emulate the Windows NT rule for display/create.	72	winnt: emulate the Windows NT rule for display/create.
73	mixed: emulate the Windows NT rule for display,	73	mixed: emulate the Windows NT rule for display,
74	emulate the Windows 95 rule for create.	74	emulate the Windows 95 rule for create.
75	Default setting is `lower'.	75	Default setting is `lower'.
76		76
77	<bool>: 0,1,yes,no,true,false	77	<bool>: 0,1,yes,no,true,false
78		78
79	TODO	79	TODO
80	----------------------------------------------------------------------	80	----------------------------------------------------------------------
81	* Need to get rid of the raw scanning stuff. Instead, always use	81	* Need to get rid of the raw scanning stuff. Instead, always use
82	a get next directory entry approach. The only thing left that uses	82	a get next directory entry approach. The only thing left that uses
83	raw scanning is the directory renaming code.	83	raw scanning is the directory renaming code.
84		84
85		85
86	POSSIBLE PROBLEMS	86	POSSIBLE PROBLEMS
87	----------------------------------------------------------------------	87	----------------------------------------------------------------------
88	* vfat_valid_longname does not properly checked reserved names.	88	* vfat_valid_longname does not properly checked reserved names.
89	* When a volume name is the same as a directory name in the root	89	* When a volume name is the same as a directory name in the root
90	directory of the filesystem, the directory name sometimes shows	90	directory of the filesystem, the directory name sometimes shows
91	up as an empty file.	91	up as an empty file.
92	* autoconv option does not work correctly.	92	* autoconv option does not work correctly.
93		93
94	BUG REPORTS	94	BUG REPORTS
95	----------------------------------------------------------------------	95	----------------------------------------------------------------------
96	If you have trouble with the VFAT filesystem, mail bug reports to	96	If you have trouble with the VFAT filesystem, mail bug reports to
97	chaffee@bmrc.cs.berkeley.edu. Please specify the filename	97	chaffee@bmrc.cs.berkeley.edu. Please specify the filename
98	and the operation that gave you trouble.	98	and the operation that gave you trouble.
99		99
100	TEST SUITE	100	TEST SUITE
101	----------------------------------------------------------------------	101	----------------------------------------------------------------------
102	If you plan to make any modifications to the vfat filesystem, please	102	If you plan to make any modifications to the vfat filesystem, please
103	get the test suite that comes with the vfat distribution at	103	get the test suite that comes with the vfat distribution at
104		104
105	http://bmrc.berkeley.edu/people/chaffee/vfat.html	105	http://bmrc.berkeley.edu/people/chaffee/vfat.html
106		106
107	This tests quite a few parts of the vfat filesystem and additional	107	This tests quite a few parts of the vfat filesystem and additional
108	tests for new features or untested features would be appreciated.	108	tests for new features or untested features would be appreciated.
109		109
110	NOTES ON THE STRUCTURE OF THE VFAT FILESYSTEM	110	NOTES ON THE STRUCTURE OF THE VFAT FILESYSTEM
111	----------------------------------------------------------------------	111	----------------------------------------------------------------------
112	(This documentation was provided by Galen C. Hunt <gchunt@cs.rochester.edu>	112	(This documentation was provided by Galen C. Hunt <gchunt@cs.rochester.edu>
113	and lightly annotated by Gordon Chaffee).	113	and lightly annotated by Gordon Chaffee).
114		114
115	This document presents a very rough, technical overview of my	115	This document presents a very rough, technical overview of my
116	knowledge of the extended FAT file system used in Windows NT 3.5 and	116	knowledge of the extended FAT file system used in Windows NT 3.5 and
117	Windows 95. I don't guarantee that any of the following is correct,	117	Windows 95. I don't guarantee that any of the following is correct,
118	but it appears to be so.	118	but it appears to be so.
119		119
120	The extended FAT file system is almost identical to the FAT	120	The extended FAT file system is almost identical to the FAT
121	file system used in DOS versions up to and including 6.223410239847	121	file system used in DOS versions up to and including 6.223410239847
122	:-). The significant change has been the addition of long file names.	122	:-). The significant change has been the addition of long file names.
123	These names support up to 255 characters including spaces and lower	123	These names support up to 255 characters including spaces and lower
124	case characters as opposed to the traditional 8.3 short names.	124	case characters as opposed to the traditional 8.3 short names.
125		125
126	Here is the description of the traditional FAT entry in the current	126	Here is the description of the traditional FAT entry in the current
127	Windows 95 filesystem:	127	Windows 95 filesystem:
128		128
129	struct directory { // Short 8.3 names	129	struct directory { // Short 8.3 names
130	unsigned char name[8]; // file name	130	unsigned char name[8]; // file name
131	unsigned char ext[3]; // file extension	131	unsigned char ext[3]; // file extension
132	unsigned char attr; // attribute byte	132	unsigned char attr; // attribute byte
133	unsigned char lcase; // Case for base and extension	133	unsigned char lcase; // Case for base and extension
134	unsigned char ctime_ms; // Creation time, milliseconds	134	unsigned char ctime_ms; // Creation time, milliseconds
135	unsigned char ctime[2]; // Creation time	135	unsigned char ctime[2]; // Creation time
136	unsigned char cdate[2]; // Creation date	136	unsigned char cdate[2]; // Creation date
137	unsigned char adate[2]; // Last access date	137	unsigned char adate[2]; // Last access date
138	unsigned char reserved[2]; // reserved values (ignored)	138	unsigned char reserved[2]; // reserved values (ignored)
139	unsigned char time[2]; // time stamp	139	unsigned char time[2]; // time stamp
140	unsigned char date[2]; // date stamp	140	unsigned char date[2]; // date stamp
141	unsigned char start[2]; // starting cluster number	141	unsigned char start[2]; // starting cluster number
142	unsigned char size[4]; // size of the file	142	unsigned char size[4]; // size of the file
143	};	143	};
144		144
145	The lcase field specifies if the base and/or the extension of an 8.3	145	The lcase field specifies if the base and/or the extension of an 8.3
146	name should be capitalized. This field does not seem to be used by	146	name should be capitalized. This field does not seem to be used by
147	Windows 95 but it is used by Windows NT. The case of filenames is not	147	Windows 95 but it is used by Windows NT. The case of filenames is not
148	completely compatible from Windows NT to Windows 95. It is not completely	148	completely compatible from Windows NT to Windows 95. It is not completely
149	compatible in the reverse direction, however. Filenames that fit in	149	compatible in the reverse direction, however. Filenames that fit in
150	the 8.3 namespace and are written on Windows NT to be lowercase will	150	the 8.3 namespace and are written on Windows NT to be lowercase will
151	show up as uppercase on Windows 95.	151	show up as uppercase on Windows 95.
152		152
153	Note that the "start" and "size" values are actually little	153	Note that the "start" and "size" values are actually little
154	endian integer values. The descriptions of the fields in this	154	endian integer values. The descriptions of the fields in this
155	structure are public knowledge and can be found elsewhere.	155	structure are public knowledge and can be found elsewhere.
156		156
157	With the extended FAT system, Microsoft has inserted extra	157	With the extended FAT system, Microsoft has inserted extra
158	directory entries for any files with extended names. (Any name which	158	directory entries for any files with extended names. (Any name which
159	legally fits within the old 8.3 encoding scheme does not have extra	159	legally fits within the old 8.3 encoding scheme does not have extra
160	entries.) I call these extra entries slots. Basically, a slot is a	160	entries.) I call these extra entries slots. Basically, a slot is a
161	specially formatted directory entry which holds up to 13 characters of	161	specially formatted directory entry which holds up to 13 characters of
162	a file's extended name. Think of slots as additional labeling for the	162	a file's extended name. Think of slots as additional labeling for the
163	directory entry of the file to which they correspond. Microsoft	163	directory entry of the file to which they correspond. Microsoft
164	prefers to refer to the 8.3 entry for a file as its alias and the	164	prefers to refer to the 8.3 entry for a file as its alias and the
165	extended slot directory entries as the file name.	165	extended slot directory entries as the file name.
166		166
167	The C structure for a slot directory entry follows:	167	The C structure for a slot directory entry follows:
168		168
169	struct slot { // Up to 13 characters of a long name	169	struct slot { // Up to 13 characters of a long name
170	unsigned char id; // sequence number for slot	170	unsigned char id; // sequence number for slot
171	unsigned char name0_4[10]; // first 5 characters in name	171	unsigned char name0_4[10]; // first 5 characters in name
172	unsigned char attr; // attribute byte	172	unsigned char attr; // attribute byte
173	unsigned char reserved; // always 0	173	unsigned char reserved; // always 0
174	unsigned char alias_checksum; // checksum for 8.3 alias	174	unsigned char alias_checksum; // checksum for 8.3 alias
175	unsigned char name5_10[12]; // 6 more characters in name	175	unsigned char name5_10[12]; // 6 more characters in name
176	unsigned char start[2]; // starting cluster number	176	unsigned char start[2]; // starting cluster number
177	unsigned char name11_12[4]; // last 2 characters in name	177	unsigned char name11_12[4]; // last 2 characters in name
178	};	178	};
179		179
180	If the layout of the slots looks a little odd, it's only	180	If the layout of the slots looks a little odd, it's only
181	because of Microsoft's efforts to maintain compatibility with old	181	because of Microsoft's efforts to maintain compatibility with old
182	software. The slots must be disguised to prevent old software from	182	software. The slots must be disguised to prevent old software from
183	panicking. To this end, a number of measures are taken:	183	panicking. To this end, a number of measures are taken:
184		184
185	1) The attribute byte for a slot directory entry is always set	185	1) The attribute byte for a slot directory entry is always set
186	to 0x0f. This corresponds to an old directory entry with	186	to 0x0f. This corresponds to an old directory entry with
187	attributes of "hidden", "system", "read-only", and "volume	187	attributes of "hidden", "system", "read-only", and "volume
188	label". Most old software will ignore any directory	188	label". Most old software will ignore any directory
189	entries with the "volume label" bit set. Real volume label	189	entries with the "volume label" bit set. Real volume label
190	entries don't have the other three bits set.	190	entries don't have the other three bits set.
191		191
192	2) The starting cluster is always set to 0, an impossible	192	2) The starting cluster is always set to 0, an impossible
193	value for a DOS file.	193	value for a DOS file.
194		194
195	Because the extended FAT system is backward compatible, it is	195	Because the extended FAT system is backward compatible, it is
196	possible for old software to modify directory entries. Measures must	196	possible for old software to modify directory entries. Measures must
197	be taken to ensure the validity of slots. An extended FAT system can	197	be taken to ensure the validity of slots. An extended FAT system can
198	verify that a slot does in fact belong to an 8.3 directory entry by	198	verify that a slot does in fact belong to an 8.3 directory entry by
199	the following:	199	the following:
200		200
201	1) Positioning. Slots for a file always immediately proceed	201	1) Positioning. Slots for a file always immediately proceed
202	their corresponding 8.3 directory entry. In addition, each	202	their corresponding 8.3 directory entry. In addition, each
203	slot has an id which marks its order in the extended file	203	slot has an id which marks its order in the extended file
204	name. Here is a very abbreviated view of an 8.3 directory	204	name. Here is a very abbreviated view of an 8.3 directory
205	entry and its corresponding long name slots for the file	205	entry and its corresponding long name slots for the file
206	"My Big File.Extension which is long":	206	"My Big File.Extension which is long":
207		207
208	<proceeding files...>	208	<proceeding files...>
209	<slot #3, id = 0x43, characters = "h is long">	209	<slot #3, id = 0x43, characters = "h is long">
210	<slot #2, id = 0x02, characters = "xtension whic">	210	<slot #2, id = 0x02, characters = "xtension whic">
211	<slot #1, id = 0x01, characters = "My Big File.E">	211	<slot #1, id = 0x01, characters = "My Big File.E">
212	<directory entry, name = "MYBIGFIL.EXT">	212	<directory entry, name = "MYBIGFIL.EXT">
213		213
214	Note that the slots are stored from last to first. Slots	214	Note that the slots are stored from last to first. Slots
215	are numbered from 1 to N. The Nth slot is or'ed with 0x40	215	are numbered from 1 to N. The Nth slot is or'ed with 0x40
216	to mark it as the last one.	216	to mark it as the last one.
217		217
218	2) Checksum. Each slot has an "alias_checksum" value. The	218	2) Checksum. Each slot has an "alias_checksum" value. The
219	checksum is calculated from the 8.3 name using the	219	checksum is calculated from the 8.3 name using the
220	following algorithm:	220	following algorithm:
221		221
222	for (sum = i = 0; i < 11; i++) {	222	for (sum = i = 0; i < 11; i++) {
223	sum = (((sum&1)<<7)\|((sum&0xfe)>>1)) + name[i]	223	sum = (((sum&1)<<7)\|((sum&0xfe)>>1)) + name[i]
224	}	224	}
225		225
226	3) If there is free space in the final slot, a Unicode NULL (0x0000)	226	3) If there is free space in the final slot, a Unicode NULL (0x0000)
227	is stored after the final character. After that, all unused	227	is stored after the final character. After that, all unused
228	characters in the final slot are set to Unicode 0xFFFF.	228	characters in the final slot are set to Unicode 0xFFFF.
229		229
230	Finally, note that the extended name is stored in Unicode. Each Unicode	230	Finally, note that the extended name is stored in Unicode. Each Unicode
231	character takes two bytes.	231	character takes two bytes.
232		232
1	EFI Real Time Clock driver	1	EFI Real Time Clock driver
2	-------------------------------	2	-------------------------------
3	S. Eranian <eranian@hpl.hp.com>	3	S. Eranian <eranian@hpl.hp.com>
4	March 2000	4	March 2000
5		5
6	I/ Introduction	6	I/ Introduction
7		7
8	This document describes the efirtc.c driver has provided for	8	This document describes the efirtc.c driver has provided for
9	the IA-64 platform.	9	the IA-64 platform.
10		10
11	The purpose of this driver is to supply an API for kernel and user applications	11	The purpose of this driver is to supply an API for kernel and user applications
12	to get access to the Time Service offered by EFI version 0.92.	12	to get access to the Time Service offered by EFI version 0.92.
13		13
14	EFI provides 4 calls one can make once the OS is booted: GetTime(),	14	EFI provides 4 calls one can make once the OS is booted: GetTime(),
15	SetTime(), GetWakeupTime(), SetWakeupTime() which are all supported by this	15	SetTime(), GetWakeupTime(), SetWakeupTime() which are all supported by this
16	driver. We describe those calls as well the design of the driver in the	16	driver. We describe those calls as well the design of the driver in the
17	following sections.	17	following sections.
18		18
19	II/ Design Decisions	19	II/ Design Decisions
20		20
21	The original ideas was to provide a very simple driver to get access to,	21	The original ideas was to provide a very simple driver to get access to,
22	at first, the time of day service. This is required in order to access, in a	22	at first, the time of day service. This is required in order to access, in a
23	portable way, the CMOS clock. A program like /sbin/hwclock uses such a clock	23	portable way, the CMOS clock. A program like /sbin/hwclock uses such a clock
24	to initialize the system view of the time during boot.	24	to initialize the system view of the time during boot.
25		25
26	Because we wanted to minimize the impact on existing user-level apps using	26	Because we wanted to minimize the impact on existing user-level apps using
27	the CMOS clock, we decided to expose an API that was very similar to the one	27	the CMOS clock, we decided to expose an API that was very similar to the one
28	used today with the legacy RTC driver (driver/char/rtc.c). However, because	28	used today with the legacy RTC driver (driver/char/rtc.c). However, because
29	EFI provides a simpler services, not all all ioctl() are available. Also	29	EFI provides a simpler services, not all ioctl() are available. Also
30	new ioctl()s have been introduced for things that EFI provides but not the	30	new ioctl()s have been introduced for things that EFI provides but not the
31	legacy.	31	legacy.
32		32
33	EFI uses a slightly different way of representing the time, noticeably	33	EFI uses a slightly different way of representing the time, noticeably
34	the reference date is different. Year is the using the full 4-digit format.	34	the reference date is different. Year is the using the full 4-digit format.
35	The Epoch is January 1st 1998. For backward compatibility reasons we don't	35	The Epoch is January 1st 1998. For backward compatibility reasons we don't
36	expose this new way of representing time. Instead we use something very	36	expose this new way of representing time. Instead we use something very
37	similar to the struct tm, i.e. struct rtc_time, as used by hwclock.	37	similar to the struct tm, i.e. struct rtc_time, as used by hwclock.
38	One of the reasons for doing it this way is to allow for EFI to still evolve	38	One of the reasons for doing it this way is to allow for EFI to still evolve
39	without necessarily impacting any of the user applications. The decoupling	39	without necessarily impacting any of the user applications. The decoupling
40	enables flexibility and permits writing wrapper code is ncase things change.	40	enables flexibility and permits writing wrapper code is ncase things change.
41		41
42	The driver exposes two interfaces, one via the device file and a set of	42	The driver exposes two interfaces, one via the device file and a set of
43	ioctl()s. The other is read-only via the /proc filesystem.	43	ioctl()s. The other is read-only via the /proc filesystem.
44		44
45	As of today we don't offer a /proc/sys interface.	45	As of today we don't offer a /proc/sys interface.
46		46
47	To allow for a uniform interface between the legacy RTC and EFI time service,	47	To allow for a uniform interface between the legacy RTC and EFI time service,
48	we have created the include/linux/rtc.h header file to contain only the	48	we have created the include/linux/rtc.h header file to contain only the
49	"public" API of the two drivers. The specifics of the legacy RTC are still	49	"public" API of the two drivers. The specifics of the legacy RTC are still
50	in include/linux/mc146818rtc.h.	50	in include/linux/mc146818rtc.h.
51		51
52		52
53	III/ Time of day service	53	III/ Time of day service
54		54
55	The part of the driver gives access to the time of day service of EFI.	55	The part of the driver gives access to the time of day service of EFI.
56	Two ioctl()s, compatible with the legacy RTC calls:	56	Two ioctl()s, compatible with the legacy RTC calls:
57		57
58	Read the CMOS clock: ioctl(d, RTC_RD_TIME, &rtc);	58	Read the CMOS clock: ioctl(d, RTC_RD_TIME, &rtc);
59		59
60	Write the CMOS clock: ioctl(d, RTC_SET_TIME, &rtc);	60	Write the CMOS clock: ioctl(d, RTC_SET_TIME, &rtc);
61		61
62	The rtc is a pointer to a data structure defined in rtc.h which is close	62	The rtc is a pointer to a data structure defined in rtc.h which is close
63	to a struct tm:	63	to a struct tm:
64		64
65	struct rtc_time {	65	struct rtc_time {
66	int tm_sec;	66	int tm_sec;
67	int tm_min;	67	int tm_min;
68	int tm_hour;	68	int tm_hour;
69	int tm_mday;	69	int tm_mday;
70	int tm_mon;	70	int tm_mon;
71	int tm_year;	71	int tm_year;
72	int tm_wday;	72	int tm_wday;
73	int tm_yday;	73	int tm_yday;
74	int tm_isdst;	74	int tm_isdst;
75	};	75	};
76		76
77	The driver takes care of converting back an forth between the EFI time and	77	The driver takes care of converting back an forth between the EFI time and
78	this format.	78	this format.
79		79
80	Those two ioctl()s can be exercised with the hwclock command:	80	Those two ioctl()s can be exercised with the hwclock command:
81		81
82	For reading:	82	For reading:
83	# /sbin/hwclock --show	83	# /sbin/hwclock --show
84	Mon Mar 6 15:32:32 2000 -0.910248 seconds	84	Mon Mar 6 15:32:32 2000 -0.910248 seconds
85		85
86	For setting:	86	For setting:
87	# /sbin/hwclock --systohc	87	# /sbin/hwclock --systohc
88		88
89	Root privileges are required to be able to set the time of day.	89	Root privileges are required to be able to set the time of day.
90		90
91	IV/ Wakeup Alarm service	91	IV/ Wakeup Alarm service
92		92
93	EFI provides an API by which one can program when a machine should wakeup,	93	EFI provides an API by which one can program when a machine should wakeup,
94	i.e. reboot. This is very different from the alarm provided by the legacy	94	i.e. reboot. This is very different from the alarm provided by the legacy
95	RTC which is some kind of interval timer alarm. For this reason we don't use	95	RTC which is some kind of interval timer alarm. For this reason we don't use
96	the same ioctl()s to get access to the service. Instead we have	96	the same ioctl()s to get access to the service. Instead we have
97	introduced 2 news ioctl()s to the interface of an RTC.	97	introduced 2 news ioctl()s to the interface of an RTC.
98		98
99	We have added 2 new ioctl()s that are specific to the EFI driver:	99	We have added 2 new ioctl()s that are specific to the EFI driver:
100		100
101	Read the current state of the alarm	101	Read the current state of the alarm
102	ioctl(d, RTC_WKLAM_RD, &wkt)	102	ioctl(d, RTC_WKLAM_RD, &wkt)
103		103
104	Set the alarm or change its status	104	Set the alarm or change its status
105	ioctl(d, RTC_WKALM_SET, &wkt)	105	ioctl(d, RTC_WKALM_SET, &wkt)
106		106
107	The wkt structure encapsulates a struct rtc_time + 2 extra fields to get	107	The wkt structure encapsulates a struct rtc_time + 2 extra fields to get
108	status information:	108	status information:
109		109
110	struct rtc_wkalrm {	110	struct rtc_wkalrm {
111		111
112	unsigned char enabled; /* =1 if alarm is enabled */	112	unsigned char enabled; /* =1 if alarm is enabled */
113	unsigned char pending; /* =1 if alarm is pending */	113	unsigned char pending; /* =1 if alarm is pending */
114		114
115	struct rtc_time time;	115	struct rtc_time time;
116	}	116	}
117		117
118	As of today, none of the existing user-level apps supports this feature.	118	As of today, none of the existing user-level apps supports this feature.
119	However writing such a program should be hard by simply using those two	119	However writing such a program should be hard by simply using those two
120	ioctl().	120	ioctl().
121		121
122	Root privileges are required to be able to set the alarm.	122	Root privileges are required to be able to set the alarm.
123		123
124	V/ References.	124	V/ References.
125		125
126	Checkout the following Web site for more information on EFI:	126	Checkout the following Web site for more information on EFI:
127		127
128	http://developer.intel.com/technology/efi/	128	http://developer.intel.com/technology/efi/
129		129
1	An ad-hoc collection of notes on IA64 MCA and INIT processing. Feel	1	An ad-hoc collection of notes on IA64 MCA and INIT processing. Feel
2	free to update it with notes about any area that is not clear.	2	free to update it with notes about any area that is not clear.
3		3
4	---	4	---
5		5
6	MCA/INIT are completely asynchronous. They can occur at any time, when	6	MCA/INIT are completely asynchronous. They can occur at any time, when
7	the OS is in any state. Including when one of the cpus is already	7	the OS is in any state. Including when one of the cpus is already
8	holding a spinlock. Trying to get any lock from MCA/INIT state is	8	holding a spinlock. Trying to get any lock from MCA/INIT state is
9	asking for deadlock. Also the state of structures that are protected	9	asking for deadlock. Also the state of structures that are protected
10	by locks is indeterminate, including linked lists.	10	by locks is indeterminate, including linked lists.
11		11
12	---	12	---
13		13
14	The complicated ia64 MCA process. All of this is mandated by Intel's	14	The complicated ia64 MCA process. All of this is mandated by Intel's
15	specification for ia64 SAL, error recovery and and unwind, it is not as	15	specification for ia64 SAL, error recovery and unwind, it is not as
16	if we have a choice here.	16	if we have a choice here.
17		17
18	* MCA occurs on one cpu, usually due to a double bit memory error.	18	* MCA occurs on one cpu, usually due to a double bit memory error.
19	This is the monarch cpu.	19	This is the monarch cpu.
20		20
21	* SAL sends an MCA rendezvous interrupt (which is a normal interrupt)	21	* SAL sends an MCA rendezvous interrupt (which is a normal interrupt)
22	to all the other cpus, the slaves.	22	to all the other cpus, the slaves.
23		23
24	* Slave cpus that receive the MCA interrupt call down into SAL, they	24	* Slave cpus that receive the MCA interrupt call down into SAL, they
25	end up spinning disabled while the MCA is being serviced.	25	end up spinning disabled while the MCA is being serviced.
26		26
27	* If any slave cpu was already spinning disabled when the MCA occurred	27	* If any slave cpu was already spinning disabled when the MCA occurred
28	then it cannot service the MCA interrupt. SAL waits ~20 seconds then	28	then it cannot service the MCA interrupt. SAL waits ~20 seconds then
29	sends an unmaskable INIT event to the slave cpus that have not	29	sends an unmaskable INIT event to the slave cpus that have not
30	already rendezvoused.	30	already rendezvoused.
31		31
32	* Because MCA/INIT can be delivered at any time, including when the cpu	32	* Because MCA/INIT can be delivered at any time, including when the cpu
33	is down in PAL in physical mode, the registers at the time of the	33	is down in PAL in physical mode, the registers at the time of the
34	event are _completely_ undefined. In particular the MCA/INIT	34	event are _completely_ undefined. In particular the MCA/INIT
35	handlers cannot rely on the thread pointer, PAL physical mode can	35	handlers cannot rely on the thread pointer, PAL physical mode can
36	(and does) modify TP. It is allowed to do that as long as it resets	36	(and does) modify TP. It is allowed to do that as long as it resets
37	TP on return. However MCA/INIT events expose us to these PAL	37	TP on return. However MCA/INIT events expose us to these PAL
38	internal TP changes. Hence curr_task().	38	internal TP changes. Hence curr_task().
39		39
40	* If an MCA/INIT event occurs while the kernel was running (not user	40	* If an MCA/INIT event occurs while the kernel was running (not user
41	space) and the kernel has called PAL then the MCA/INIT handler cannot	41	space) and the kernel has called PAL then the MCA/INIT handler cannot
42	assume that the kernel stack is in a fit state to be used. Mainly	42	assume that the kernel stack is in a fit state to be used. Mainly
43	because PAL may or may not maintain the stack pointer internally.	43	because PAL may or may not maintain the stack pointer internally.
44	Because the MCA/INIT handlers cannot trust the kernel stack, they	44	Because the MCA/INIT handlers cannot trust the kernel stack, they
45	have to use their own, per-cpu stacks. The MCA/INIT stacks are	45	have to use their own, per-cpu stacks. The MCA/INIT stacks are
46	preformatted with just enough task state to let the relevant handlers	46	preformatted with just enough task state to let the relevant handlers
47	do their job.	47	do their job.
48		48
49	* Unlike most other architectures, the ia64 struct task is embedded in	49	* Unlike most other architectures, the ia64 struct task is embedded in
50	the kernel stack[1]. So switching to a new kernel stack means that	50	the kernel stack[1]. So switching to a new kernel stack means that
51	we switch to a new task as well. Because various bits of the kernel	51	we switch to a new task as well. Because various bits of the kernel
52	assume that current points into the struct task, switching to a new	52	assume that current points into the struct task, switching to a new
53	stack also means a new value for current.	53	stack also means a new value for current.
54		54
55	* Once all slaves have rendezvoused and are spinning disabled, the	55	* Once all slaves have rendezvoused and are spinning disabled, the
56	monarch is entered. The monarch now tries to diagnose the problem	56	monarch is entered. The monarch now tries to diagnose the problem
57	and decide if it can recover or not.	57	and decide if it can recover or not.
58		58
59	* Part of the monarch's job is to look at the state of all the other	59	* Part of the monarch's job is to look at the state of all the other
60	tasks. The only way to do that on ia64 is to call the unwinder,	60	tasks. The only way to do that on ia64 is to call the unwinder,
61	as mandated by Intel.	61	as mandated by Intel.
62		62
63	* The starting point for the unwind depends on whether a task is	63	* The starting point for the unwind depends on whether a task is
64	running or not. That is, whether it is on a cpu or is blocked. The	64	running or not. That is, whether it is on a cpu or is blocked. The
65	monarch has to determine whether or not a task is on a cpu before it	65	monarch has to determine whether or not a task is on a cpu before it
66	knows how to start unwinding it. The tasks that received an MCA or	66	knows how to start unwinding it. The tasks that received an MCA or
67	INIT event are no longer running, they have been converted to blocked	67	INIT event are no longer running, they have been converted to blocked
68	tasks. But (and its a big but), the cpus that received the MCA	68	tasks. But (and its a big but), the cpus that received the MCA
69	rendezvous interrupt are still running on their normal kernel stacks!	69	rendezvous interrupt are still running on their normal kernel stacks!
70		70
71	* To distinguish between these two cases, the monarch must know which	71	* To distinguish between these two cases, the monarch must know which
72	tasks are on a cpu and which are not. Hence each slave cpu that	72	tasks are on a cpu and which are not. Hence each slave cpu that
73	switches to an MCA/INIT stack, registers its new stack using	73	switches to an MCA/INIT stack, registers its new stack using
74	set_curr_task(), so the monarch can tell that the _original_ task is	74	set_curr_task(), so the monarch can tell that the _original_ task is
75	no longer running on that cpu. That gives us a decent chance of	75	no longer running on that cpu. That gives us a decent chance of
76	getting a valid backtrace of the _original_ task.	76	getting a valid backtrace of the _original_ task.
77		77
78	* MCA/INIT can be nested, to a depth of 2 on any cpu. In the case of a	78	* MCA/INIT can be nested, to a depth of 2 on any cpu. In the case of a
79	nested error, we want diagnostics on the MCA/INIT handler that	79	nested error, we want diagnostics on the MCA/INIT handler that
80	failed, not on the task that was originally running. Again this	80	failed, not on the task that was originally running. Again this
81	requires set_curr_task() so the MCA/INIT handlers can register their	81	requires set_curr_task() so the MCA/INIT handlers can register their
82	own stack as running on that cpu. Then a recursive error gets a	82	own stack as running on that cpu. Then a recursive error gets a
83	trace of the failing handler's "task".	83	trace of the failing handler's "task".
84		84
85	[1] My (Keith Owens) original design called for ia64 to separate its	85	[1] My (Keith Owens) original design called for ia64 to separate its
86	struct task and the kernel stacks. Then the MCA/INIT data would be	86	struct task and the kernel stacks. Then the MCA/INIT data would be
87	chained stacks like i386 interrupt stacks. But that required	87	chained stacks like i386 interrupt stacks. But that required
88	radical surgery on the rest of ia64, plus extra hard wired TLB	88	radical surgery on the rest of ia64, plus extra hard wired TLB
89	entries with its associated performance degradation. David	89	entries with its associated performance degradation. David
90	Mosberger vetoed that approach. Which meant that separate kernel	90	Mosberger vetoed that approach. Which meant that separate kernel
91	stacks meant separate "tasks" for the MCA/INIT handlers.	91	stacks meant separate "tasks" for the MCA/INIT handlers.
92		92
93	---	93	---
94		94
95	INIT is less complicated than MCA. Pressing the nmi button or using	95	INIT is less complicated than MCA. Pressing the nmi button or using
96	the equivalent command on the management console sends INIT to all	96	the equivalent command on the management console sends INIT to all
97	cpus. SAL picks one one of the cpus as the monarch and the rest are	97	cpus. SAL picks one of the cpus as the monarch and the rest are
98	slaves. All the OS INIT handlers are entered at approximately the same	98	slaves. All the OS INIT handlers are entered at approximately the same
99	time. The OS monarch prints the state of all tasks and returns, after	99	time. The OS monarch prints the state of all tasks and returns, after
100	which the slaves return and the system resumes.	100	which the slaves return and the system resumes.
101		101
102	At least that is what is supposed to happen. Alas there are broken	102	At least that is what is supposed to happen. Alas there are broken
103	versions of SAL out there. Some drive all the cpus as monarchs. Some	103	versions of SAL out there. Some drive all the cpus as monarchs. Some
104	drive them all as slaves. Some drive one cpu as monarch, wait for that	104	drive them all as slaves. Some drive one cpu as monarch, wait for that
105	cpu to return from the OS then drive the rest as slaves. Some versions	105	cpu to return from the OS then drive the rest as slaves. Some versions
106	of SAL cannot even cope with returning from the OS, they spin inside	106	of SAL cannot even cope with returning from the OS, they spin inside
107	SAL on resume. The OS INIT code has workarounds for some of these	107	SAL on resume. The OS INIT code has workarounds for some of these
108	broken SAL symptoms, but some simply cannot be fixed from the OS side.	108	broken SAL symptoms, but some simply cannot be fixed from the OS side.
109		109
110	---	110	---
111		111
112	The scheduler hooks used by ia64 (curr_task, set_curr_task) are layer	112	The scheduler hooks used by ia64 (curr_task, set_curr_task) are layer
113	violations. Unfortunately MCA/INIT start off as massive layer	113	violations. Unfortunately MCA/INIT start off as massive layer
114	violations (can occur at _any_ time) and they build from there.	114	violations (can occur at _any_ time) and they build from there.
115		115
116	At least ia64 makes an attempt at recovering from hardware errors, but	116	At least ia64 makes an attempt at recovering from hardware errors, but
117	it is a difficult problem because of the asynchronous nature of these	117	it is a difficult problem because of the asynchronous nature of these
118	errors. When processing an unmaskable interrupt we sometimes need	118	errors. When processing an unmaskable interrupt we sometimes need
119	special code to cope with our inability to take any locks.	119	special code to cope with our inability to take any locks.
120		120
121	---	121	---
122		122
123	How is ia64 MCA/INIT different from x86 NMI?	123	How is ia64 MCA/INIT different from x86 NMI?
124		124
125	* x86 NMI typically gets delivered to one cpu. MCA/INIT gets sent to	125	* x86 NMI typically gets delivered to one cpu. MCA/INIT gets sent to
126	all cpus.	126	all cpus.
127		127
128	* x86 NMI cannot be nested. MCA/INIT can be nested, to a depth of 2	128	* x86 NMI cannot be nested. MCA/INIT can be nested, to a depth of 2
129	per cpu.	129	per cpu.
130		130
131	* x86 has a separate struct task which points to one of multiple kernel	131	* x86 has a separate struct task which points to one of multiple kernel
132	stacks. ia64 has the struct task embedded in the single kernel	132	stacks. ia64 has the struct task embedded in the single kernel
133	stack, so switching stack means switching task.	133	stack, so switching stack means switching task.
134		134
135	* x86 does not call the BIOS so the NMI handler does not have to worry	135	* x86 does not call the BIOS so the NMI handler does not have to worry
136	about any registers having changed. MCA/INIT can occur while the cpu	136	about any registers having changed. MCA/INIT can occur while the cpu
137	is in PAL in physical mode, with undefined registers and an undefined	137	is in PAL in physical mode, with undefined registers and an undefined
138	kernel stack.	138	kernel stack.
139		139
140	* i386 backtrace is not very sensitive to whether a process is running	140	* i386 backtrace is not very sensitive to whether a process is running
141	or not. ia64 unwind is very, very sensitive to whether a process is	141	or not. ia64 unwind is very, very sensitive to whether a process is
142	running or not.	142	running or not.
143		143
144	---	144	---
145		145
146	What happens when MCA/INIT is delivered what a cpu is running user	146	What happens when MCA/INIT is delivered what a cpu is running user
147	space code?	147	space code?
148		148
149	The user mode registers are stored in the RSE area of the MCA/INIT on	149	The user mode registers are stored in the RSE area of the MCA/INIT on
150	entry to the OS and are restored from there on return to SAL, so user	150	entry to the OS and are restored from there on return to SAL, so user
151	mode registers are preserved across a recoverable MCA/INIT. Since the	151	mode registers are preserved across a recoverable MCA/INIT. Since the
152	OS has no idea what unwind data is available for the user space stack,	152	OS has no idea what unwind data is available for the user space stack,
153	MCA/INIT never tries to backtrace user space. Which means that the OS	153	MCA/INIT never tries to backtrace user space. Which means that the OS
154	does not bother making the user space process look like a blocked task,	154	does not bother making the user space process look like a blocked task,
155	i.e. the OS does not copy pt_regs and switch_stack to the user space	155	i.e. the OS does not copy pt_regs and switch_stack to the user space
156	stack. Also the OS has no idea how big the user space RSE and memory	156	stack. Also the OS has no idea how big the user space RSE and memory
157	stacks are, which makes it too risky to copy the saved state to a user	157	stacks are, which makes it too risky to copy the saved state to a user
158	mode stack.	158	mode stack.
159		159
160	---	160	---
161		161
162	How do we get a backtrace on the tasks that were running when MCA/INIT	162	How do we get a backtrace on the tasks that were running when MCA/INIT
163	was delivered?	163	was delivered?
164		164
165	mca.c:::ia64_mca_modify_original_stack(). That identifies and	165	mca.c:::ia64_mca_modify_original_stack(). That identifies and
166	verifies the original kernel stack, copies the dirty registers from	166	verifies the original kernel stack, copies the dirty registers from
167	the MCA/INIT stack's RSE to the original stack's RSE, copies the	167	the MCA/INIT stack's RSE to the original stack's RSE, copies the
168	skeleton struct pt_regs and switch_stack to the original stack, fills	168	skeleton struct pt_regs and switch_stack to the original stack, fills
169	in the skeleton structures from the PAL minstate area and updates the	169	in the skeleton structures from the PAL minstate area and updates the
170	original stack's thread.ksp. That makes the original stack look	170	original stack's thread.ksp. That makes the original stack look
171	exactly like any other blocked task, i.e. it now appears to be	171	exactly like any other blocked task, i.e. it now appears to be
172	sleeping. To get a backtrace, just start with thread.ksp for the	172	sleeping. To get a backtrace, just start with thread.ksp for the
173	original task and unwind like any other sleeping task.	173	original task and unwind like any other sleeping task.
174		174
175	---	175	---
176		176
177	How do we identify the tasks that were running when MCA/INIT was	177	How do we identify the tasks that were running when MCA/INIT was
178	delivered?	178	delivered?
179		179
180	If the previous task has been verified and converted to a blocked	180	If the previous task has been verified and converted to a blocked
181	state, then sos->prev_task on the MCA/INIT stack is updated to point to	181	state, then sos->prev_task on the MCA/INIT stack is updated to point to
182	the previous task. You can look at that field in dumps or debuggers.	182	the previous task. You can look at that field in dumps or debuggers.
183	To help distinguish between the handler and the original tasks,	183	To help distinguish between the handler and the original tasks,
184	handlers have _TIF_MCA_INIT set in thread_info.flags.	184	handlers have _TIF_MCA_INIT set in thread_info.flags.
185		185
186	The sos data is always in the MCA/INIT handler stack, at offset	186	The sos data is always in the MCA/INIT handler stack, at offset
187	MCA_SOS_OFFSET. You can get that value from mca_asm.h or calculate it	187	MCA_SOS_OFFSET. You can get that value from mca_asm.h or calculate it
188	as KERNEL_STACK_SIZE - sizeof(struct pt_regs) - sizeof(struct	188	as KERNEL_STACK_SIZE - sizeof(struct pt_regs) - sizeof(struct
189	ia64_sal_os_state), with 16 byte alignment for all structures.	189	ia64_sal_os_state), with 16 byte alignment for all structures.
190		190
191	Also the comm field of the MCA/INIT task is modified to include the pid	191	Also the comm field of the MCA/INIT task is modified to include the pid
192	of the original task, for humans to use. For example, a comm field of	192	of the original task, for humans to use. For example, a comm field of
193	'MCA 12159' means that pid 12159 was running when the MCA was	193	'MCA 12159' means that pid 12159 was running when the MCA was
194	delivered.	194	delivered.
195		195
1	$Id: INTERFACE.fax,v 1.2 2000/08/06 09:22:50 armin Exp $	1	$Id: INTERFACE.fax,v 1.2 2000/08/06 09:22:50 armin Exp $
2		2
3		3
4	Description of the fax-subinterface between linklevel and hardwarelevel of	4	Description of the fax-subinterface between linklevel and hardwarelevel of
5	isdn4linux.	5	isdn4linux.
6		6
7	The communication between linklevel (LL) and hardwarelevel (HL) for fax	7	The communication between linklevel (LL) and hardwarelevel (HL) for fax
8	is based on the struct T30_s (defined in isdnif.h).	8	is based on the struct T30_s (defined in isdnif.h).
9	This struct is allocated in the LL.	9	This struct is allocated in the LL.
10	In order to use fax, the LL provides the pointer to this struct with the	10	In order to use fax, the LL provides the pointer to this struct with the
11	command ISDN_CMD_SETL3 (parm.fax). This pointer expires in case of hangup	11	command ISDN_CMD_SETL3 (parm.fax). This pointer expires in case of hangup
12	and when a new channel to a new connection is assigned.	12	and when a new channel to a new connection is assigned.
13		13
14		14
15	Data handling:	15	Data handling:
16	In send-mode the HL-driver has to handle the <DLE> codes and the bit-order	16	In send-mode the HL-driver has to handle the <DLE> codes and the bit-order
17	conversion by itself.	17	conversion by itself.
18	In receive-mode the LL-driver takes care of the bit-order conversion	18	In receive-mode the LL-driver takes care of the bit-order conversion
19	(specified by +FBOR)	19	(specified by +FBOR)
20		20
21	Structure T30_s description:	21	Structure T30_s description:
22		22
23	This structure stores the values (set by AT-commands), the remote-	23	This structure stores the values (set by AT-commands), the remote-
24	capability-values and the command-codes between LL and HL.	24	capability-values and the command-codes between LL and HL.
25		25
26	If the HL-driver receives ISDN_CMD_FAXCMD, all needed information	26	If the HL-driver receives ISDN_CMD_FAXCMD, all needed information
27	is in this struct set by the LL.	27	is in this struct set by the LL.
28	To signal information to the LL, the HL-driver has to set the	28	To signal information to the LL, the HL-driver has to set the
29	the parameters and use ISDN_STAT_FAXIND.	29	parameters and use ISDN_STAT_FAXIND.
30	(Please refer to INTERFACE)	30	(Please refer to INTERFACE)
31		31
32	Structure T30_s:	32	Structure T30_s:
33		33
34	All members are 8-bit unsigned (__u8)	34	All members are 8-bit unsigned (__u8)
35		35
36	- resolution	36	- resolution
37	- rate	37	- rate
38	- width	38	- width
39	- length	39	- length
40	- compression	40	- compression
41	- ecm	41	- ecm
42	- binary	42	- binary
43	- scantime	43	- scantime
44	- id[]	44	- id[]
45	Local faxmachine's parameters, set by +FDIS, +FDCS, +FLID, ...	45	Local faxmachine's parameters, set by +FDIS, +FDCS, +FLID, ...
46		46
47	- r_resolution	47	- r_resolution
48	- r_rate	48	- r_rate
49	- r_width	49	- r_width
50	- r_length	50	- r_length
51	- r_compression	51	- r_compression
52	- r_ecm	52	- r_ecm
53	- r_binary	53	- r_binary
54	- r_scantime	54	- r_scantime
55	- r_id[]	55	- r_id[]
56	Remote faxmachine's parameters. To be set by HL-driver.	56	Remote faxmachine's parameters. To be set by HL-driver.
57		57
58	- phase	58	- phase
59	Defines the actual state of fax connection. Set by HL or LL	59	Defines the actual state of fax connection. Set by HL or LL
60	depending on progress and type of connection.	60	depending on progress and type of connection.
61	If the phase changes because of an AT command, the LL driver	61	If the phase changes because of an AT command, the LL driver
62	changes this value. Otherwise the HL-driver takes care of it, but	62	changes this value. Otherwise the HL-driver takes care of it, but
63	only necessary on call establishment (from IDLE to PHASE_A).	63	only necessary on call establishment (from IDLE to PHASE_A).
64	(one of the constants ISDN_FAX_PHASE_[IDLE,A,B,C,D,E])	64	(one of the constants ISDN_FAX_PHASE_[IDLE,A,B,C,D,E])
65		65
66	- direction	66	- direction
67	Defines outgoing/send or incoming/receive connection.	67	Defines outgoing/send or incoming/receive connection.
68	(ISDN_TTY_FAX_CONN_[IN,OUT])	68	(ISDN_TTY_FAX_CONN_[IN,OUT])
69		69
70	- code	70	- code
71	Commands from LL to HL; possible constants :	71	Commands from LL to HL; possible constants :
72	ISDN_TTY_FAX_DR signals +FDR command to HL	72	ISDN_TTY_FAX_DR signals +FDR command to HL
73		73
74	ISDN_TTY_FAX_DT signals +FDT command to HL	74	ISDN_TTY_FAX_DT signals +FDT command to HL
75		75
76	ISDN_TTY_FAX_ET signals +FET command to HL	76	ISDN_TTY_FAX_ET signals +FET command to HL
77		77
78		78
79	Other than that the "code" is set with the hangup-code value at	79	Other than that the "code" is set with the hangup-code value at
80	the end of connection for the +FHNG message.	80	the end of connection for the +FHNG message.
81		81
82	- r_code	82	- r_code
83	Commands from HL to LL; possible constants :	83	Commands from HL to LL; possible constants :
84	ISDN_TTY_FAX_CFR output of +FCFR message.	84	ISDN_TTY_FAX_CFR output of +FCFR message.
85		85
86	ISDN_TTY_FAX_RID output of remote ID set in r_id[]	86	ISDN_TTY_FAX_RID output of remote ID set in r_id[]
87	(+FCSI/+FTSI on send/receive)	87	(+FCSI/+FTSI on send/receive)
88		88
89	ISDN_TTY_FAX_DCS output of +FDCS and CONNECT message,	89	ISDN_TTY_FAX_DCS output of +FDCS and CONNECT message,
90	switching to phase C.	90	switching to phase C.
91		91
92	ISDN_TTY_FAX_ET signals end of data,	92	ISDN_TTY_FAX_ET signals end of data,
93	switching to phase D.	93	switching to phase D.
94		94
95	ISDN_TTY_FAX_FCON signals the established, outgoing connection,	95	ISDN_TTY_FAX_FCON signals the established, outgoing connection,
96	switching to phase B.	96	switching to phase B.
97		97
98	ISDN_TTY_FAX_FCON_I signals the established, incoming connection,	98	ISDN_TTY_FAX_FCON_I signals the established, incoming connection,
99	switching to phase B.	99	switching to phase B.
100		100
101	ISDN_TTY_FAX_DIS output of +FDIS message and values.	101	ISDN_TTY_FAX_DIS output of +FDIS message and values.
102		102
103	ISDN_TTY_FAX_SENT signals that all data has been sent	103	ISDN_TTY_FAX_SENT signals that all data has been sent
104	and <DLE><ETX> is acknowledged,	104	and <DLE><ETX> is acknowledged,
105	OK message will be sent.	105	OK message will be sent.
106		106
107	ISDN_TTY_FAX_PTS signals a msg-confirmation (page sent successful),	107	ISDN_TTY_FAX_PTS signals a msg-confirmation (page sent successful),
108	depending on fet value:	108	depending on fet value:
109	0: output OK message (more pages follow)	109	0: output OK message (more pages follow)
110	1: switching to phase B (next document)	110	1: switching to phase B (next document)
111		111
112	ISDN_TTY_FAX_TRAIN_OK output of +FDCS and OK message (for receive mode).	112	ISDN_TTY_FAX_TRAIN_OK output of +FDCS and OK message (for receive mode).
113		113
114	ISDN_TTY_FAX_EOP signals end of data in receive mode,	114	ISDN_TTY_FAX_EOP signals end of data in receive mode,
115	switching to phase D.	115	switching to phase D.
116		116
117	ISDN_TTY_FAX_HNG output of the +FHNG and value set by code and	117	ISDN_TTY_FAX_HNG output of the +FHNG and value set by code and
118	OK message, switching to phase E.	118	OK message, switching to phase E.
119		119
120		120
121	- badlin	121	- badlin
122	Value of +FBADLIN	122	Value of +FBADLIN
123		123
124	- badmul	124	- badmul
125	Value of +FBADMUL	125	Value of +FBADMUL
126		126
127	- bor	127	- bor
128	Value of +FBOR	128	Value of +FBOR
129		129
130	- fet	130	- fet
131	Value of +FET command in send-mode.	131	Value of +FET command in send-mode.
132	Set by HL in receive-mode for +FET message.	132	Set by HL in receive-mode for +FET message.
133		133
134	- pollid[]	134	- pollid[]
135	ID-string, set by +FCIG	135	ID-string, set by +FCIG
136		136
137	- cq	137	- cq
138	Value of +FCQ	138	Value of +FCQ
139		139
140	- cr	140	- cr
141	Value of +FCR	141	Value of +FCR
142		142
143	- ctcrty	143	- ctcrty
144	Value of +FCTCRTY	144	Value of +FCTCRTY
145		145
146	- minsp	146	- minsp
147	Value of +FMINSP	147	Value of +FMINSP
148		148
149	- phcto	149	- phcto
150	Value of +FPHCTO	150	Value of +FPHCTO
151		151
152	- rel	152	- rel
153	Value of +FREL	153	Value of +FREL
154		154
155	- nbc	155	- nbc
156	Value of +FNBC (0,1)	156	Value of +FNBC (0,1)
157	(+FNBC is not a known class 2 fax command, I added this to change the	157	(+FNBC is not a known class 2 fax command, I added this to change the
158	automatic "best capabilities" connection in the eicon HL-driver)	158	automatic "best capabilities" connection in the eicon HL-driver)
159		159
160		160
161	Armin	161	Armin
162	mac@melware.de	162	mac@melware.de
163		163
164		164
1	$Id: README.hysdn,v 1.3.6.1 2001/02/10 14:41:19 kai Exp $	1	$Id: README.hysdn,v 1.3.6.1 2001/02/10 14:41:19 kai Exp $
2	The hysdn driver has been written by	2	The hysdn driver has been written by
3	by Werner Cornelius (werner@isdn4linux.de or werner@titro.de)	3	Werner Cornelius (werner@isdn4linux.de or werner@titro.de)
4	for Hypercope GmbH Aachen Germany. Hypercope agreed to publish this driver	4	for Hypercope GmbH Aachen Germany. Hypercope agreed to publish this driver
5	under the GNU General Public License.	5	under the GNU General Public License.
6		6
7	The CAPI 2.0-support was added by Ulrich Albrecht (ualbrecht@hypercope.de)	7	The CAPI 2.0-support was added by Ulrich Albrecht (ualbrecht@hypercope.de)
8	for Hypercope GmbH Aachen, Germany.	8	for Hypercope GmbH Aachen, Germany.
9		9
10		10
11	This program is free software; you can redistribute it and/or modify	11	This program is free software; you can redistribute it and/or modify
12	it under the terms of the GNU General Public License as published by	12	it under the terms of the GNU General Public License as published by
13	the Free Software Foundation; either version 2 of the License, or	13	the Free Software Foundation; either version 2 of the License, or
14	(at your option) any later version.	14	(at your option) any later version.
15		15
16	This program is distributed in the hope that it will be useful,	16	This program is distributed in the hope that it will be useful,
17	but WITHOUT ANY WARRANTY; without even the implied warranty of	17	but WITHOUT ANY WARRANTY; without even the implied warranty of
18	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the	18	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19	GNU General Public License for more details.	19	GNU General Public License for more details.
20		20
21	You should have received a copy of the GNU General Public License	21	You should have received a copy of the GNU General Public License
22	along with this program; if not, write to the Free Software	22	along with this program; if not, write to the Free Software
23	Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.	23	Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
24		24
25	Table of contents	25	Table of contents
26	=================	26	=================
27		27
28	1. About the driver	28	1. About the driver
29		29
30	2. Loading/Unloading the driver	30	2. Loading/Unloading the driver
31		31
32	3. Entries in the /proc filesystem	32	3. Entries in the /proc filesystem
33		33
34	4. The /proc/net/hysdn/cardconfX file	34	4. The /proc/net/hysdn/cardconfX file
35		35
36	5. The /proc/net/hysdn/cardlogX file	36	5. The /proc/net/hysdn/cardlogX file
37		37
38	6. Where to get additional info and help	38	6. Where to get additional info and help
39		39
40		40
41	1. About the driver	41	1. About the driver
42		42
43	The drivers/isdn/hysdn subdir contains a driver for HYPERCOPEs active	43	The drivers/isdn/hysdn subdir contains a driver for HYPERCOPEs active
44	PCI isdn cards Champ, Ergo and Metro. To enable support for this cards	44	PCI isdn cards Champ, Ergo and Metro. To enable support for this cards
45	enable ISDN support in the kernel config and support for HYSDN cards in	45	enable ISDN support in the kernel config and support for HYSDN cards in
46	the active cards submenu. The driver may only be compiled and used if	46	the active cards submenu. The driver may only be compiled and used if
47	support for loadable modules and the process filesystem have been enabled.	47	support for loadable modules and the process filesystem have been enabled.
48		48
49	These cards provide two different interfaces to the kernel. Without the	49	These cards provide two different interfaces to the kernel. Without the
50	optional CAPI 2.0 support, they register as ethernet card. IP-routing	50	optional CAPI 2.0 support, they register as ethernet card. IP-routing
51	to a ISDN-destination is performed on the card itself. All necessary	51	to a ISDN-destination is performed on the card itself. All necessary
52	handlers for various protocols like ppp and others as well as config info	52	handlers for various protocols like ppp and others as well as config info
53	and firmware may be fetched from Hypercopes WWW-Site www.hypercope.de.	53	and firmware may be fetched from Hypercopes WWW-Site www.hypercope.de.
54		54
55	With CAPI 2.0 support enabled, the card can also be used as a CAPI 2.0	55	With CAPI 2.0 support enabled, the card can also be used as a CAPI 2.0
56	compliant devices with either CAPI 2.0 applications	56	compliant devices with either CAPI 2.0 applications
57	(check isdn4k-utils) or -using the capidrv module- as a regular	57	(check isdn4k-utils) or -using the capidrv module- as a regular
58	isdn4linux device. This is done via the same mechanism as with the	58	isdn4linux device. This is done via the same mechanism as with the
59	active AVM cards and in fact uses the same module.	59	active AVM cards and in fact uses the same module.
60		60
61		61
62	2. Loading/Unloading the driver	62	2. Loading/Unloading the driver
63		63
64	The module has no command line parameters and auto detects up to 10 cards	64	The module has no command line parameters and auto detects up to 10 cards
65	in the id-range 0-9.	65	in the id-range 0-9.
66	If a loaded driver shall be unloaded all open files in the /proc/net/hysdn	66	If a loaded driver shall be unloaded all open files in the /proc/net/hysdn
67	subdir need to be closed and all ethernet interfaces allocated by this	67	subdir need to be closed and all ethernet interfaces allocated by this
68	driver must be shut down. Otherwise the module counter will avoid a module	68	driver must be shut down. Otherwise the module counter will avoid a module
69	unload.	69	unload.
70		70
71	If you are using the CAPI 2.0-interface, make sure to load/modprobe the	71	If you are using the CAPI 2.0-interface, make sure to load/modprobe the
72	kernelcapi-module first.	72	kernelcapi-module first.
73		73
74	If you plan to use the capidrv-link to isdn4linux, make sure to load	74	If you plan to use the capidrv-link to isdn4linux, make sure to load
75	capidrv.o after all modules using this driver (i.e. after hysdn and	75	capidrv.o after all modules using this driver (i.e. after hysdn and
76	any avm-specific modules).	76	any avm-specific modules).
77		77
78	3. Entries in the /proc filesystem	78	3. Entries in the /proc filesystem
79		79
80	When the module has been loaded it adds the directory hysdn in the	80	When the module has been loaded it adds the directory hysdn in the
81	/proc/net tree. This directory contains exactly 2 file entries for each	81	/proc/net tree. This directory contains exactly 2 file entries for each
82	card. One is called cardconfX and the other cardlogX, where X is the	82	card. One is called cardconfX and the other cardlogX, where X is the
83	card id number from 0 to 9.	83	card id number from 0 to 9.
84	The cards are numbered in the order found in the PCI config data.	84	The cards are numbered in the order found in the PCI config data.
85		85
86	4. The /proc/net/hysdn/cardconfX file	86	4. The /proc/net/hysdn/cardconfX file
87		87
88	This file may be read to get by everyone to get info about the cards type,	88	This file may be read to get by everyone to get info about the cards type,
89	actual state, available features and used resources.	89	actual state, available features and used resources.
90	The first 3 entries (id, bus and slot) are PCI info fields, the following	90	The first 3 entries (id, bus and slot) are PCI info fields, the following
91	type field gives the information about the cards type:	91	type field gives the information about the cards type:
92		92
93	4 -> Ergo card (server card with 2 b-chans)	93	4 -> Ergo card (server card with 2 b-chans)
94	5 -> Metro card (server card with 4 or 8 b-chans)	94	5 -> Metro card (server card with 4 or 8 b-chans)
95	6 -> Champ card (client card with 2 b-chans)	95	6 -> Champ card (client card with 2 b-chans)
96		96
97	The following 3 fields show the hardware assignments for irq, iobase and the	97	The following 3 fields show the hardware assignments for irq, iobase and the
98	dual ported memory (dp-mem).	98	dual ported memory (dp-mem).
99	The fields b-chans and fax-chans announce the available card resources of	99	The fields b-chans and fax-chans announce the available card resources of
100	this types for the user.	100	this types for the user.
101	The state variable indicates the actual drivers state for this card with the	101	The state variable indicates the actual drivers state for this card with the
102	following assignments.	102	following assignments.
103		103
104	0 -> card has not been booted since driver load	104	0 -> card has not been booted since driver load
105	1 -> card booting is actually in progess	105	1 -> card booting is actually in progess
106	2 -> card is in an error state due to a previous boot failure	106	2 -> card is in an error state due to a previous boot failure
107	3 -> card is booted and active	107	3 -> card is booted and active
108		108
109	And the last field (device) shows the name of the ethernet device assigned	109	And the last field (device) shows the name of the ethernet device assigned
110	to this card. Up to the first successful boot this field only shows a -	110	to this card. Up to the first successful boot this field only shows a -
111	to tell that no net device has been allocated up to now. Once a net device	111	to tell that no net device has been allocated up to now. Once a net device
112	has been allocated it remains assigned to this card, even if a card is	112	has been allocated it remains assigned to this card, even if a card is
113	rebooted and an boot error occurs.	113	rebooted and an boot error occurs.
114		114
115	Writing to the cardconfX file boots the card or transfers config lines to	115	Writing to the cardconfX file boots the card or transfers config lines to
116	the cards firmware. The type of data is automatically detected when the	116	the cards firmware. The type of data is automatically detected when the
117	first data is written. Only root has write access to this file.	117	first data is written. Only root has write access to this file.
118	The firmware boot files are normally called hyclient.pof for client cards	118	The firmware boot files are normally called hyclient.pof for client cards
119	and hyserver.pof for server cards.	119	and hyserver.pof for server cards.
120	After successfully writing the boot file, complete config files or single	120	After successfully writing the boot file, complete config files or single
121	config lines may be copied to this file.	121	config lines may be copied to this file.
122	If an error occurs the return value given to the writing process has the	122	If an error occurs the return value given to the writing process has the
123	following additional codes (decimal):	123	following additional codes (decimal):
124		124
125	1000 Another process is currently bootng the card	125	1000 Another process is currently bootng the card
126	1001 Invalid firmware header	126	1001 Invalid firmware header
127	1002 Boards dual-port RAM test failed	127	1002 Boards dual-port RAM test failed
128	1003 Internal firmware handler error	128	1003 Internal firmware handler error
129	1004 Boot image size invalid	129	1004 Boot image size invalid
130	1005 First boot stage (bootstrap loader) failed	130	1005 First boot stage (bootstrap loader) failed
131	1006 Second boot stage failure	131	1006 Second boot stage failure
132	1007 Timeout waiting for card ready during boot	132	1007 Timeout waiting for card ready during boot
133	1008 Operation only allowed in booted state	133	1008 Operation only allowed in booted state
134	1009 Config line too long	134	1009 Config line too long
135	1010 Invalid channel number	135	1010 Invalid channel number
136	1011 Timeout sending config data	136	1011 Timeout sending config data
137		137
138	Additional info about error reasons may be fetched from the log output.	138	Additional info about error reasons may be fetched from the log output.
139		139
140	5. The /proc/net/hysdn/cardlogX file	140	5. The /proc/net/hysdn/cardlogX file
141		141
142	The cardlogX file entry may be opened multiple for reading by everyone to	142	The cardlogX file entry may be opened multiple for reading by everyone to
143	get the cards and drivers log data. Card messages always start with the	143	get the cards and drivers log data. Card messages always start with the
144	keyword LOG. All other lines are output from the driver.	144	keyword LOG. All other lines are output from the driver.
145	The driver log data may be redirected to the syslog by selecting the	145	The driver log data may be redirected to the syslog by selecting the
146	appropriate bitmask. The cards log messages will always be send to this	146	appropriate bitmask. The cards log messages will always be send to this
147	interface but never to the syslog.	147	interface but never to the syslog.
148		148
149	A root user may write a decimal or hex (with 0x) value t this file to select	149	A root user may write a decimal or hex (with 0x) value t this file to select
150	desired output options. As mentioned above the cards log dat is always	150	desired output options. As mentioned above the cards log dat is always
151	written to the cardlog file independent of the following options only used	151	written to the cardlog file independent of the following options only used
152	to check and debug the driver itself:	152	to check and debug the driver itself:
153		153
154	For example:	154	For example:
155	echo "0x34560078" > /proc/net/hysdn/cardlog0	155	echo "0x34560078" > /proc/net/hysdn/cardlog0
156	to output the hex log mask 34560078 for card 0.	156	to output the hex log mask 34560078 for card 0.
157		157
158	The written value is regarded as an unsigned 32-Bit value, bit ored for	158	The written value is regarded as an unsigned 32-Bit value, bit ored for
159	desired output. The following bits are already assigned:	159	desired output. The following bits are already assigned:
160		160
161	0x80000000 All driver log data is alternatively via syslog	161	0x80000000 All driver log data is alternatively via syslog
162	0x00000001 Log memory allocation errors	162	0x00000001 Log memory allocation errors
163	0x00000010 Firmware load start and close are logged	163	0x00000010 Firmware load start and close are logged
164	0x00000020 Log firmware record parser	164	0x00000020 Log firmware record parser
165	0x00000040 Log every firmware write actions	165	0x00000040 Log every firmware write actions
166	0x00000080 Log all card related boot messages	166	0x00000080 Log all card related boot messages
167	0x00000100 Output all config data sent for debugging purposes	167	0x00000100 Output all config data sent for debugging purposes
168	0x00000200 Only non comment config lines are shown wth channel	168	0x00000200 Only non comment config lines are shown wth channel
169	0x00000400 Additional conf log output	169	0x00000400 Additional conf log output
170	0x00001000 Log the asynchronous scheduler actions (config and log)	170	0x00001000 Log the asynchronous scheduler actions (config and log)
171	0x00100000 Log all open and close actions to /proc/net/hysdn/card files	171	0x00100000 Log all open and close actions to /proc/net/hysdn/card files
172	0x00200000 Log all actions from /proc file entries	172	0x00200000 Log all actions from /proc file entries
173	0x00010000 Log network interface init and deinit	173	0x00010000 Log network interface init and deinit
174		174
175	6. Where to get additional info and help	175	6. Where to get additional info and help
176		176
177	If you have any problems concerning the driver or configuration contact	177	If you have any problems concerning the driver or configuration contact
178	the Hypercope support team (support@hypercope.de) and or the authors	178	the Hypercope support team (support@hypercope.de) and or the authors
179	Werner Cornelius (werner@isdn4linux or cornelius@titro.de) or	179	Werner Cornelius (werner@isdn4linux or cornelius@titro.de) or
180	Ulrich Albrecht (ualbrecht@hypercope.de).	180	Ulrich Albrecht (ualbrecht@hypercope.de).
181		181
182		182
183		183
184		184
185		185
186		186
187		187
188		188
189		189
190		190
191		191
192		192
193		193
194		194
195		195
196		196
1	Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.	1	Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.
2		2
3	Contents	3	Contents
4	=======	4	=======
5	- 1. Introduction	5	- 1. Introduction
6	- 2. Identifying the adapter/interface	6	- 2. Identifying the adapter/interface
7	- 3. Features supported	7	- 3. Features supported
8	- 4. Command line parameters	8	- 4. Command line parameters
9	- 5. Performance suggestions	9	- 5. Performance suggestions
10	- 6. Available Downloads	10	- 6. Available Downloads
11		11
12		12
13	1. Introduction:	13	1. Introduction:
14	This Linux driver supports Neterion's Xframe I PCI-X 1.0 and	14	This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
15	Xframe II PCI-X 2.0 adapters. It supports several features	15	Xframe II PCI-X 2.0 adapters. It supports several features
16	such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.	16	such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
17	See below for complete list of features.	17	See below for complete list of features.
18	All features are supported for both IPv4 and IPv6.	18	All features are supported for both IPv4 and IPv6.
19		19
20	2. Identifying the adapter/interface:	20	2. Identifying the adapter/interface:
21	a. Insert the adapter(s) in your system.	21	a. Insert the adapter(s) in your system.
22	b. Build and load driver	22	b. Build and load driver
23	# insmod s2io.ko	23	# insmod s2io.ko
24	c. View log messages	24	c. View log messages
25	# dmesg \| tail -40	25	# dmesg \| tail -40
26	You will see messages similar to:	26	You will see messages similar to:
27	eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA	27	eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
28	eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA	28	eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
29	eth4: Device is on 64 bit 133MHz PCIX(M1) bus	29	eth4: Device is on 64 bit 133MHz PCIX(M1) bus
30		30
31	The above messages identify the adapter type(Xframe I/II), adapter revision,	31	The above messages identify the adapter type(Xframe I/II), adapter revision,
32	driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).	32	driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
33	In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed	33	In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
34	as well.	34	as well.
35		35
36	To associate an interface with a physical adapter use "ethtool -p <ethX>".	36	To associate an interface with a physical adapter use "ethtool -p <ethX>".
37	The corresponding adapter's LED will blink multiple times.	37	The corresponding adapter's LED will blink multiple times.
38		38
39	3. Features supported:	39	3. Features supported:
40	a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes,	40	a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes,
41	modifiable using ifconfig command.	41	modifiable using ifconfig command.
42		42
43	b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit	43	b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
44	and receive, TSO.	44	and receive, TSO.
45		45
46	c. Multi-buffer receive mode. Scattering of packet across multiple	46	c. Multi-buffer receive mode. Scattering of packet across multiple
47	buffers. Currently driver supports 2-buffer mode which yields	47	buffers. Currently driver supports 2-buffer mode which yields
48	significant performance improvement on certain platforms(SGI Altix,	48	significant performance improvement on certain platforms(SGI Altix,
49	IBM xSeries).	49	IBM xSeries).
50		50
51	d. MSI/MSI-X. Can be enabled on platforms which support this feature	51	d. MSI/MSI-X. Can be enabled on platforms which support this feature
52	(IA64, Xeon) resulting in noticeable performance improvement(upto 7%	52	(IA64, Xeon) resulting in noticeable performance improvement(upto 7%
53	on certain platforms).	53	on certain platforms).
54		54
55	e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt	55	e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt
56	moderation.	56	moderation.
57		57
58	f. Statistics. Comprehensive MAC-level and software statistics displayed	58	f. Statistics. Comprehensive MAC-level and software statistics displayed
59	using "ethtool -S" option.	59	using "ethtool -S" option.
60		60
61	g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,	61	g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
62	with multiple steering options.	62	with multiple steering options.
63		63
64	4. Command line parameters	64	4. Command line parameters
65	a. tx_fifo_num	65	a. tx_fifo_num
66	Number of transmit queues	66	Number of transmit queues
67	Valid range: 1-8	67	Valid range: 1-8
68	Default: 1	68	Default: 1
69		69
70	b. rx_ring_num	70	b. rx_ring_num
71	Number of receive rings	71	Number of receive rings
72	Valid range: 1-8	72	Valid range: 1-8
73	Default: 1	73	Default: 1
74		74
75	c. tx_fifo_len	75	c. tx_fifo_len
76	Size of each transmit queue	76	Size of each transmit queue
77	Valid range: Total length of all queues should not exceed 8192	77	Valid range: Total length of all queues should not exceed 8192
78	Default: 4096	78	Default: 4096
79		79
80	d. rx_ring_sz	80	d. rx_ring_sz
81	Size of each receive ring(in 4K blocks)	81	Size of each receive ring(in 4K blocks)
82	Valid range: Limited by memory on system	82	Valid range: Limited by memory on system
83	Default: 30	83	Default: 30
84		84
85	e. intr_type	85	e. intr_type
86	Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X)	86	Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X)
87	Valid range: 1-3	87	Valid range: 1-3
88	Default: 1	88	Default: 1
89		89
90	5. Performance suggestions	90	5. Performance suggestions
91	General:	91	General:
92	a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)	92	a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
93	b. Set TCP windows size to optimal value.	93	b. Set TCP windows size to optimal value.
94	For instance, for MTU=1500 a value of 210K has been observed to result in	94	For instance, for MTU=1500 a value of 210K has been observed to result in
95	good performance.	95	good performance.
96	# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"	96	# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
97	# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"	97	# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
98	For MTU=9000, TCP window size of 10 MB is recommended.	98	For MTU=9000, TCP window size of 10 MB is recommended.
99	# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"	99	# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
100	# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"	100	# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
101		101
102	Transmit performance:	102	Transmit performance:
103	a. By default, the driver respects BIOS settings for PCI bus parameters.	103	a. By default, the driver respects BIOS settings for PCI bus parameters.
104	However, you may want to experiment with PCI bus parameters	104	However, you may want to experiment with PCI bus parameters
105	max-split-transactions(MOST) and MMRBC (use setpci command).	105	max-split-transactions(MOST) and MMRBC (use setpci command).
106	A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.	106	A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.
107	It could be different for your hardware.	107	It could be different for your hardware.
108	Set MMRBC to 4K**.	108	Set MMRBC to 4K**.
109		109
110	For example you can set	110	For example you can set
111	For opteron	111	For opteron
112	#setpci -d 17d5:* 62=1d	112	#setpci -d 17d5:* 62=1d
113	For Itanium	113	For Itanium
114	#setpci -d 17d5:* 62=3d	114	#setpci -d 17d5:* 62=3d
115		115
116	For detailed description of the PCI registers, please see Xframe User Guide.	116	For detailed description of the PCI registers, please see Xframe User Guide.
117		117
118	b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this	118	b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this
119	parameter.	119	parameter.
120	c. Turn on TSO(using "ethtool -K")	120	c. Turn on TSO(using "ethtool -K")
121	# ethtool -K <ethX> tso on	121	# ethtool -K <ethX> tso on
122		122
123	Receive performance:	123	Receive performance:
124	a. By default, the driver respects BIOS settings for PCI bus parameters.	124	a. By default, the driver respects BIOS settings for PCI bus parameters.
125	However, you may want to set PCI latency timer to 248.	125	However, you may want to set PCI latency timer to 248.
126	#setpci -d 17d5:* LATENCY_TIMER=f8	126	#setpci -d 17d5:* LATENCY_TIMER=f8
127	For detailed description of the PCI registers, please see Xframe User Guide.	127	For detailed description of the PCI registers, please see Xframe User Guide.
128	b. Use 2-buffer mode. This results in large performance boost on	128	b. Use 2-buffer mode. This results in large performance boost on
129	on certain platforms(eg. SGI Altix, IBM xSeries).	129	certain platforms(eg. SGI Altix, IBM xSeries).
130	c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to	130	c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to
131	set/verify this option.	131	set/verify this option.
132	d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network	132	d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network
133	device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to	133	device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to
134	bring down CPU utilization.	134	bring down CPU utilization.
135		135
136	** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are	136	** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are
137	recommended as safe parameters.	137	recommended as safe parameters.
138	For more information, please review the AMD8131 errata at	138	For more information, please review the AMD8131 errata at
139	http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf	139	http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf
140		140
141	6. Available Downloads	141	6. Available Downloads
142	Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up	142	Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up
143	to date, also the latest "s2io" code (including support for 2.4 kernels) is	143	to date, also the latest "s2io" code (including support for 2.4 kernels) is
144	available via "Support" link on the Neterion site: http://www.neterion.com.	144	available via "Support" link on the Neterion site: http://www.neterion.com.
145		145
146	For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com,	146	For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com,
147	user: linuxdocs password: HALdocs	147	user: linuxdocs password: HALdocs
148		148
149	7. Support	149	7. Support
150	For further support please contact either your 10GbE Xframe NIC vendor (IBM,	150	For further support please contact either your 10GbE Xframe NIC vendor (IBM,
151	HP, SGI etc.) or click on the "Support" link on the Neterion site:	151	HP, SGI etc.) or click on the "Support" link on the Neterion site:
152	http://www.neterion.com.	152	http://www.neterion.com.
153		153
154		154