Doug / smarc-fsl-linux-kernel

21 Jan, 2009

1 commit

06 Jan, 2009

1 commit

12 Dec, 2008

1 commit

  • 17d217fe9   Btrfs: fix nodatasum handling in balancing code ... Browse Code »

    Checksums on data can be disabled by mount option, so it's
    possible some data extents don't have checksums or have
    invalid checksums. This causes trouble for data relocation.
    This patch contains following things to make data relocation
    work.

    1) make nodatasum/nodatacow mount option only affects new
    files. Checksums and COW on data are only controlled by the
    inode flags.

    2) check the existence of checksum in the nodatacow checker.
    If checksums exist, force COW the data extent. This ensure that
    checksum for a given block is either valid or does not exist.

    3) update data relocation code to properly handle the case
    of checksum missing.

    Signed-off-by: Yan Zheng

    Yan Zheng
     

09 Dec, 2008

1 commit

  • d20f7043f   Btrfs: move data checksumming into a dedicated tree ... Browse Code »

    Btrfs stores checksums for each data block. Until now, they have
    been stored in the subvolume trees, indexed by the inode that is
    referencing the data block. This means that when we read the inode,
    we've probably read in at least some checksums as well.

    But, this has a few problems:

    * The checksums are indexed by logical offset in the file. When
    compression is on, this means we have to do the expensive checksumming
    on the uncompressed data. It would be faster if we could checksum
    the compressed data instead.

    * If we implement encryption, we'll be checksumming the plain text and
    storing that on disk. This is significantly less secure.

    * For either compression or encryption, we have to get the plain text
    back before we can verify the checksum as correct. This makes the raid
    layer balancing and extent moving much more expensive.

    * It makes the front end caching code more complex, as we have touch
    the subvolume and inodes as we cache extents.

    * There is potentitally one copy of the checksum in each subvolume
    referencing an extent.

    The solution used here is to store the extent checksums in a dedicated
    tree. This allows us to index the checksums by phyiscal extent
    start and length. It means:

    * The checksum is against the data stored on disk, after any compression
    or encryption is done.

    * The checksum is stored in a central location, and can be verified without
    following back references, or reading inodes.

    This makes compression significantly faster by reducing the amount of
    data that needs to be checksummed. It will also allow much faster
    raid management code in general.

    The checksums are indexed by a key with a fixed objectid (a magic value
    in ctree.h) and offset set to the starting byte of the extent. This
    allows us to copy the checksum items into the fsync log tree directly (or
    any other tree), without having to invent a second format for them.

    Signed-off-by: Chris Mason

    Chris Mason
     

20 Nov, 2008

2 commits

11 Nov, 2008

2 commits

10 Nov, 2008

1 commit

  • ff5b7ee33   Btrfs: Fix csum error for compressed data ... Browse Code »

    The decompress code doesn't take the logical offset in extent
    pointer into account. If the logical offset isn't zero, data
    will be decompressed into wrong pages.

    The solution used here is to record the starting offset of the extent
    in the file separately from the logical start of the extent_map struct.
    This allows us to avoid problems inserting overlapping extents.

    Signed-off-by: Yan Zheng

    Yan Zheng
     

08 Nov, 2008

1 commit

  • af09abfec   Btrfs: make sure compressed bios don't complete too soon ... Browse Code »

    When writing a compressed extent, a number of bios are created that
    point to a single struct compressed_bio. At end_io time an atomic counter in
    the compressed_bio struct makes sure that all of the bios have finished
    before final end_io processing is done.

    But when multiple bios are needed to write a compressed extent, the
    counter was being incremented after the first bio was sent to submit_bio.
    It is possible the bio will complete before the counter is incremented,
    making the end_io handler free the compressed_bio struct before
    processing is finished.

    The fix is to increment the atomic counter before bio submission,
    both for compressed reads and writes.

    Signed-off-by: Chris Mason

    Chris Mason
     

07 Nov, 2008

1 commit

  • 771ed689d   Btrfs: Optimize compressed writeback and reads ... Browse Code »

    When reading compressed extents, try to put pages into the page cache
    for any pages covered by the compressed extent that readpages didn't already
    preload.

    Add an async work queue to handle transformations at delayed allocation processing
    time. Right now this is just compression. The workflow is:

    1) Find offsets in the file marked for delayed allocation
    2) Lock the pages
    3) Lock the state bits
    4) Call the async delalloc code

    The async delalloc code clears the state lock bits and delalloc bits. It is
    important this happens before the range goes into the work queue because
    otherwise it might deadlock with other work queue items that try to lock
    those extent bits.

    The file pages are compressed, and if the compression doesn't work the
    pages are written back directly.

    An ordered work queue is used to make sure the inodes are written in the same
    order that pdflush or writepages sent them down.

    This changes extent_write_cache_pages to let the writepage function
    update the wbc nr_written count.

    Signed-off-by: Chris Mason

    Chris Mason
     

01 Nov, 2008

1 commit

  • 70b99e695   Btrfs: Compression corner fixes ... Browse Code »

    Make sure we keep page->mapping NULL on the pages we're getting
    via alloc_page. It gets set so a few of the callbacks can do the right
    thing, but in general these pages don't have a mapping.

    Don't try to truncate compressed inline items in btrfs_drop_extents.
    The whole compressed item must be preserved.

    Don't try to create multipage inline compressed items. When we try to
    overwrite just the first page of the file, we would have to read in and recow
    all the pages after it in the same compressed inline items. For now, only
    create single page inline items.

    Make sure we lock pages in the correct order during delalloc. The
    search into the state tree for delalloc bytes can return bytes before
    the page we already have locked.

    Signed-off-by: Chris Mason

    Chris Mason
     

31 Oct, 2008

1 commit

30 Oct, 2008

1 commit

  • c8b978188   Btrfs: Add zlib compression support ... Browse Code »

    This is a large change for adding compression on reading and writing,
    both for inline and regular extents. It does some fairly large
    surgery to the writeback paths.

    Compression is off by default and enabled by mount -o compress. Even
    when the -o compress mount option is not used, it is possible to read
    compressed extents off the disk.

    If compression for a given set of pages fails to make them smaller, the
    file is flagged to avoid future compression attempts later.

    * While finding delalloc extents, the pages are locked before being sent down
    to the delalloc handler. This allows the delalloc handler to do complex things
    such as cleaning the pages, marking them writeback and starting IO on their
    behalf.

    * Inline extents are inserted at delalloc time now. This allows us to compress
    the data before inserting the inline extent, and it allows us to insert
    an inline extent that spans multiple pages.

    * All of the in-memory extent representations (extent_map.c, ordered-data.c etc)
    are changed to record both an in-memory size and an on disk size, as well
    as a flag for compression.

    From a disk format point of view, the extent pointers in the file are changed
    to record the on disk size of a given extent and some encoding flags.
    Space in the disk format is allocated for compression encoding, as well
    as encryption and a generic 'other' field. Neither the encryption or the
    'other' field are currently used.

    In order to limit the amount of data read for a single random read in the
    file, the size of a compressed extent is limited to 128k. This is a
    software only limit, the disk format supports u64 sized compressed extents.

    In order to limit the ram consumed while processing extents, the uncompressed
    size of a compressed extent is limited to 256k. This is a software only limit
    and will be subject to tuning later.

    Checksumming is still done on compressed extents, and it is done on the
    uncompressed version of the data. This way additional encodings can be
    layered on without having to figure out which encoding to checksum.

    Compression happens at delalloc time, which is basically singled threaded because
    it is usually done by a single pdflush thread. This makes it tricky to
    spread the compression load across all the cpus on the box. We'll have to
    look at parallel pdflush walks of dirty inodes at a later time.

    Decompression is hooked into readpages and it does spread across CPUs nicely.

    Signed-off-by: Chris Mason

    Chris Mason