11 Sep, 2013
1 commit
-
This is the recovery side of the btree block owner change operation
performed by swapext on CRC enabled filesystems. We detect that an
owner change is needed by the flag that has been placed on the inode
log format flag field. Because the inode recovery is being replayed
after the buffers that make up the BMBT in the given checkpoint, we
can walk all the buffers and directly modify them when we see the
flag set on an inode.Because the inode can be relogged and hence present in multiple
chekpoints with the "change owner" flag set, we could do multiple
passes across the inode to do this change. While this isn't optimal,
we can't directly ignore the flag as there may be multiple
independent swap extent operations being replayed on the same inode
in different checkpoints so we can't ignore them.Further, because the owner change operation uses ordered buffers, we
might have buffers that are newer on disk than the current
checkpoint and so already have the owner changed in them. Hence we
cannot just peek at a buffer in the tree and check that it has the
correct owner and assume that the change was completed.So, for the moment just brute force the owner change every time we
see an inode with the flag set. Note that we have to be careful here
because the owner of the buffers may point to either the old owner
or the new owner. Currently the verifier can't verify the owner
directly, so there is no failure case here right now. If we verify
the owner exactly in future, then we'll have to take this into
account.This was tested in terms of normal operation via xfstests - all of
the fsr tests now pass without failure. however, we really need to
modify xfs/227 to stress v3 inodes correctly to ensure we fully
cover this case for v5 filesystems.In terms of recovery testing, I used a hacked version of xfs_fsr
that held the temp inode open for a few seconds before exiting so
that the filesystem could be shut down with an open owner change
recovery flags set on at least the temp inode. fsr leaves the temp
inode unlinked and in btree format, so this was necessary for the
owner change to be reliably replayed.logprint confirmed the tmp inode in the log had the correct flag set:
INO: cnt:3 total:3 a:0x69e9e0 len:56 a:0x69ea20 len:176 a:0x69eae0 len:88
INODE: #regs:3 ino:0x44 flags:0x209 dsize:88
^^^^^0x200 is set, indicating a data fork owner change needed to be
replayed on inode 0x44. A printk in the revoery code confirmed that
the inode change was recovered:XFS (vdc): Mounting Filesystem
XFS (vdc): Starting recovery (logdev: internal)
recovering owner change ino 0x44
XFS (vdc): Version 5 superblock detected. This kernel L support enabled!
Use of these features in this kernel is at your own risk!
XFS (vdc): Ending recovery (logdev: internal)The script used to test this was:
$ cat ./recovery-fsr.sh
#!/bin/bashdev=/dev/vdc
mntpt=/mnt/scratch
testfile=$mntpt/testfileumount $mntpt
mkfs.xfs -f -m crc=1 $dev
mount $dev $mntpt
chmod 777 $mntptfor i in `seq 10000 -1 0`; do
xfs_io -f -d -c "pwrite $(($i * 4096)) 4096" $testfile > /dev/null 2>&1
done
xfs_bmap -vp $testfile |head -20xfs_fsr -d -v $testfile &
sleep 10
/home/dave/src/xfstests-dev/src/godown -f $mntpt
wait
umount $mntptxfs_logprint -t $dev |tail -20
time mount $dev $mntpt
xfs_bmap -vp $testfile
umount $mntpt
$Signed-off-by: Dave Chinner
Reviewed-by: Mark Tinguely
Signed-off-by: Ben Myers
10 Sep, 2013
1 commit
-
For CRC enabled filesystems, we can't just swap inode forks from one
inode to another when defragmenting a file - the blocks in the inode
fork bmap btree contain pointers back to the owner inode. Hence if
we are to swap the inode forks we have to atomically modify every
block in the btree during the transaction.We are doing an entire fork swap here, so we could create a new
transaction item type that indicates we are changing the owner of a
certain structure from one value to another. If we combine this with
ordered buffer logging to modify all the buffers in the tree, then
we can change the buffers in the tree without needing log space for
the operation. However, this then requires log recovery to perform
the modification of the owner information of the objects/structures
in question.This does introduce some interesting ordering details into recovery:
we have to make sure that the owner change replay occurs after the
change that moves the objects is made, not before. Hence we can't
use a separate log item for this as we have no guarantee of strict
ordering between multiple items in the log due to the relogging
action of asynchronous transaction commits. Hence there is no
"generic" method we can use for changing the ownership of arbitrary
metadata structures.For inode forks, however, there is a simple method of communicating
that the fork contents need the owner rewritten - we can pass a
inode log format flag for the fork for the transaction that does a
fork swap. This flag will then follow the inode fork through
relogging actions so when the swap actually gets replayed the
ownership can be changed immediately by log recovery. So that gives
us a simple method of "whole fork" exchange between two inodes.This is relatively simple to implement, so it makes sense to do this
as an initial implementation to support xfs_fsr on CRC enabled
filesytems in the same manner as we do on existing filesystems. This
commit introduces the swapext driven functionality, the recovery
functionality will be in a separate patch.Signed-off-by: Dave Chinner
Reviewed-by: Mark Tinguely
Signed-off-by: Ben Myers
13 Aug, 2013
1 commit
-
Little things like exported functions, __KERNEL__ protections, and
so on that ensure user and kernel shared headers are identical.Signed-off-by: Dave Chinner
Reviewed-by: Brian Foster
Reviewed-by: Mark Tinguely
Signed-off-by: Ben Myers
08 May, 2013
1 commit
-
Running a CONFIG_XFS_DEBUG kernel in production environments is not
the best idea as it introduces significant overhead, can change
the behaviour of algorithms (such as allocation) to improve test
coverage, and (most importantly) panic the machine on non-fatal
errors.There are many cases where all we want to do is run a
kernel with more bounds checking enabled, such as is provided by the
ASSERT() statements throughout the code, but without all the
potential overhead and drawbacks.This patch converts all the ASSERT statements to evaluate as
WARN_ON(1) statements and hence if they fail dump a warning and a
stack trace to the log. This has minimal overhead and does not
change any algorithms, and will allow us to find strange "out of
bounds" problems more easily on production machines.There are a few places where assert statements contain debug only
code. These are converted to be debug-or-warn only code so that we
still get all the assert checks in the code.Signed-off-by: Dave Chinner
Reviewed-by: Brian Foster
Signed-off-by: Ben Myers
22 Apr, 2013
1 commit
-
Add support for larger btree blocks that contains a CRC32C checksum,
a filesystem uuid and block number for detecting filesystem
consistency and out of place writes.[dchinner@redhat.com] Also include an owner field to allow reverse
mappings to be implemented for improved repairability and a LSN
field to so that log recovery can easily determine the last
modification that made it to disk for each buffer.[dchinner@redhat.com] Add buffer log format flags to indicate the
type of buffer to recovery so that we don't have to do blind magic
number tests to determine what the buffer is.[dchinner@redhat.com] Modified to fit into the verifier structure.
Signed-off-by: Christoph Hellwig
Signed-off-by: Dave Chinner
Reviewed-by: Ben Myers
Signed-off-by: Ben Myers
16 Nov, 2012
3 commits
-
To separate the verifiers from iodone functions and associate read
and write verifiers at the same time, introduce a buffer verifier
operations structure to the xfs_buf.This avoids the need for assigning the write verifier, clearing the
iodone function and re-running ioend processing in the read
verifier, and gets rid of the nasty "b_pre_io" name for the write
verifier function pointer. If we ever need to, it will also be
easier to add further content specific callbacks to a buffer with an
ops structure in place.We also avoid needing to export verifier functions, instead we
can simply export the ops structures for those that are needed
outside the function they are defined in.This patch also fixes a directory block readahead verifier issue
it exposed.This patch also adds ops callbacks to the inode/alloc btree blocks
initialised by growfs. These will need more work before they will
work with CRCs.Signed-off-by: Dave Chinner
Reviewed-by: Phil White
Signed-off-by: Ben Myers -
Metadata buffers that are read from disk have write verifiers
already attached to them, but newly allocated buffers do not. Add
appropriate write verifiers to all new metadata buffers.Signed-off-by: Dave Chinner
Reviewed-by: Ben Myers
Signed-off-by: Ben Myers -
Add an btree block verify callback function and pass it into the
buffer read functions. Because each different btree block type
requires different verification, add a function to the ops structure
that is called from the generic code.Also, propagate the verification callback functions through the
readahead functions, and into the external bmap and bulkstat inode
readahead code that uses the generic btree buffer read functions.Signed-off-by: Dave Chinner
Reviewed-by: Phil White
Signed-off-by: Ben Myers
14 Nov, 2012
1 commit
-
Factor xfs_btree_init_block() to be independent of the btree cursor,
and use the function to initialise btree blocks in the growfs code.
This makes adding support for different format btree blocks simple.Signed-off-by: Dave Chinner
Reviewed-by Rich Johnston
Signed-off-by: Ben Myers
26 Jul, 2011
1 commit
-
Remove the definition and usages of the macro XFS_BUF_PTR.
Signed-off-by: Chandra Seetharaman
Reviewed-by: Christoph Hellwig
Signed-off-by: Alex Elder
13 Jul, 2011
1 commit
-
Remove various bits left over from the old kdb-only btree tracing code, but
leave the actual trace point stubs in place to ease adding new event based
btree tracing.Signed-off-by: Christoph Hellwig
Reviewed-by: Alex Elder
Reviewed-by: Dave Chinner
19 Oct, 2010
1 commit
-
The implementation os ->kill_root only differ by either simply
zeroing out the now unused buffer in the btree cursor in the inode
allocation btree or using xfs_btree_setbuf in the allocation btree.Initially both of them used xfs_btree_setbuf, but the use in the
ialloc btree was removed early on because it interacted badly with
xfs_trans_binval.In addition to zeroing out the buffer in the cursor xfs_btree_setbuf
updates the bc_ra array in the btree cursor, and calls
xfs_trans_brelse on the buffer previous occupying the slot.The bc_ra update should be done for the alloc btree updated too,
although the lack of it does not cause serious problems. The
xfs_trans_brelse call on the other hand is effectively a no-op in
the end - it keeps decrementing the bli_recur refcount until it hits
zero, and then just skips out because the buffer will always be
dirty at this point. So removing it for the allocation btree is
just fine.So unify the code and move it to xfs_btree.c. While we're at it
also replace the call to xfs_btree_setbuf with a NULL bp argument in
xfs_btree_del_cursor with a direct call to xfs_trans_brelse given
that the cursor is beeing freed just after this and the state
updates are superflous. After this xfs_btree_setbuf is only used
with a non-NULL bp argument and can thus be simplified.Signed-off-by: Christoph Hellwig
Signed-off-by: Alex Elder
01 Sep, 2009
1 commit
-
A lot more functions could be made static, but they need
forward declarations; this does some easy ones, and also
found a few unused functions in the process.Signed-off-by: Eric Sandeen
Reviewed-by: Christoph Hellwig
Signed-off-by: Felix Blyakher
29 Mar, 2009
1 commit
-
Signed-off-by: Malcolm Parsons
Reviewed-by: Christoph Hellwig
30 Oct, 2008
26 commits
-
structures.
Always use the generic xfs_btree_block type instead of the short / long
structures. Add XFS_BTREE_SBLOCK_LEN / XFS_BTREE_LBLOCK_LEN defines for
the length of a short / long form block. The rationale for this is that we
will grow more btree block header variants to support CRCs and other RAS
information, and always accessing them through the same datatype with
unions for the short / long form pointers makes implementing this much
easier.SGI-PV: 988146
SGI-Modid: xfs-linux-melb:xfs-kern:32300a
Signed-off-by: Christoph Hellwig
Signed-off-by: Donald Douwsma
Signed-off-by: David Chinner
Signed-off-by: Lachlan McIlroy -
Replace the generic record / key / ptr addressing macros that use cpp
token pasting with simpler macros that do the job for just one given btree
type. The new macros lose the cur argument and thus can be used outside
the core btree code, but also gain an xfs_mount * argument to allow for
checking the CRC flag in the near future. Note that many of these macros
aren't actually used in the kernel code, but only in userspace (mostly in
xfs_repair).SGI-PV: 988146
SGI-Modid: xfs-linux-melb:xfs-kern:32295a
Signed-off-by: Christoph Hellwig
Signed-off-by: Donald Douwsma
Signed-off-by: David Chinner
Signed-off-by: Lachlan McIlroy -
Clean up the way the maximum and minimum records for the btree blocks are
calculated. For the alloc and inobt btrees all the values are
pre-calculated in xfs_mount_common, and we switch the current loop around
the ugly generic macros that use cpp token pasting to generate type names
to two small helpers in normal C code. For the bmbt and bmdr trees these
helpers also exist, but can be called during runtime, too. Here we also
kill various macros dealing with them and inline the logic into the
get_minrecs / get_maxrecs / get_dmaxrecs methods in xfs_bmap_btree.c.Note that all these new helpers take an xfs_mount * argument which will be
needed to determine the size of a btree block once we add support for
extended btree blocks with CRCs and other RAS information.SGI-PV: 988146
SGI-Modid: xfs-linux-melb:xfs-kern:32292a
Signed-off-by: Christoph Hellwig
Signed-off-by: Donald Douwsma
Signed-off-by: Lachlan McIlroy -
SGI-PV: 986558
SGI-Modid: xfs-linux-melb:xfs-kern:32231a
Signed-off-by: Barry Naujok
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy -
Lots of functionality in xfs_btree.c isn't needed by callers outside of
this file anymore, so mark these functions static.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32209a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Add methods to check whether two keys/records are in the righ order. This
replaces the xfs_btree_check_key and xfs_btree_check_rec methods. For the
callers from xfs_bmap.c just opencode the bmbt-specific asserts.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32208a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
These are equivalent to the xfs_btree_* versions, and the only remaining
caller can be switched to the generic one after they are exported. Also
remove some now dead infrastructure in xfs_bmap_btree.c.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32207a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Not really much reason to make it generic given that it's so small, but
this is the last non-method in xfs_alloc_btree.c and xfs_ialloc_btree.c,
so it makes the whole btree implementation more structured.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32206a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the btree delete code generic. Based on a patch from David Chinner
with lots of changes to follow the original btree implementations more
closely. While this loses some of the generic helper routines for
inserting/moving/removing records it also solves some of the one off bugs
in the original code and makes it easier to verify.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32205a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
xfs_bmbt_killroot is a mostly generic implementation of moving from a real
block based root to an inode based root. So move it to xfs_btree.c where
it can use all the nice infrastructure there and make it pointer size
agnosticThe new name for it is xfs_btree_kill_iroot, following the old naming but
making it clear we're dealing with the root in inode case here, and to
avoid confusion with xfs_btree_new_root which is used for the not inode
rooted case. I've also added a comment describing what it does and why
it's named the way it is.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32203a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the btree insert code generic. Based on a patch from David Chinner
with lots of changes to follow the original btree implementations more
closely. While this loses some of the generic helper routines for
inserting/moving/removing records it also solves some of the one off bugs
in the original code and makes it easier to verify.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32202a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
xfs_bmbt_newroot is a mostly generic implementation of moving from an
inode root to a real block based root. So move it to xfs_btree.c where it
can use all the nice infrastructure there and make it pointer size
agnosticThe new name for it is xfs_btree_new_iroot, following the old naming but
making it clear we're dealing with the root in inode case here, and to
avoid confusion with xfs_btree_new_root which is used for the not inode
rooted case.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32201a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
Add a xfs_btree_new_root helper for the alloc and ialloc btrees. The bmap
btree needs it's own version and is not converted.[hch: split out from bigger patch and minor adaptions]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32200a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the btree split code generic. Based on a patch from David Chinner
with lots of changes to follow the original btree implementations more
closely. While this loses some of the generic helper routines for
inserting/moving/removing records it also solves some of the one off bugs
in the original code and makes it easier to verify.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32198a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the btree left shift code generic. Based on a patch from David
Chinner with lots of changes to follow the original btree implementations
more closely. While this loses some of the generic helper routines for
inserting/moving/removing records it also solves some of the one off bugs
in the original code and makes it easier to verify.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32197a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the btree right shift code generic. Based on a patch from David
Chinner with lots of changes to follow the original btree implementations
more closely. While this loses some of the generic helper routines for
inserting/moving/removing records it also solves some of the one off bugs
in the original code and makes it easier to verify.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32196a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
The most complicated part here is the lastrec tracking for the alloc
btree. Most logic is in the update_lastrec method which has to do some
hopefully good enough dirty magic to maintain it.[hch: split out from bigger patch and a rework of the lastrec
logic]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32194a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
Note that there are many > 80 char lines introduced due to the
xfs_btree_key casts. But the places where this happens is throw-away code
once the whole btree code gets merged into a common implementation.The same is true for the temporary xfs_alloc_log_keys define to the new
name. All old users will be gone after a few patches.[hch: split out from bigger patch and minor adaptions]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32193a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
[hch: split out from bigger patch and minor adaptions]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32192a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
[hch: split out from bigger patch and minor adaptions]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32191a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
Because this is the first major generic btree routine this patch includes
some infrastrucure, first a few routines to deal with a btree block that
can be either in short or long form, second xfs_btree_read_buf_block,
which is the new central routine to read a btree block given a cursor, and
third the new xfs_btree_ptr_addr routine to calculate the address for a
given btree pointer record.[hch: split out from bigger patch and minor adaptions]
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32190a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Add new helpers in xfs_btree.c to find the record, key and block pointer
entries inside a btree block. To implement this genericly the
->get_maxrecs methods and two new xfs_btree_ops entries for the key and
record sizes are used. Also add a big comment describing how the
addressing inside a btree block works.Note that these helpers are unused until users are introduced in the next
patches and this patch will thus cause some harmless compiler warnings.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32189a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Factor xfs_btree_maxrecs into a per-btree operation.
The get_maxrecs method is based on a patch from Dave Chinner.
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32188a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
Make the existing bmap btree tracing generic so that it applies to all
btree types.Some fragments lifted from a patch by Dave Chinner.
SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32187a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner -
From: Dave Chinner
Introduce statistics coverage of all the btrees and cover all the btree
operations, not just some.Invaluable for determining test code coverage of all the btree
operations....SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32184a
Signed-off-by: David Chinner
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell -
Move the various btree validation helpers around in xfs_btree.c so that
they are close to each other and in common #ifdef DEBUG sections.Also add a new xfs_btree_check_ptr helper to check a btree ptr that can be
either long or short form.Split out from a bigger patch from Dave Chinner with various small changes
applied by me.SGI-PV: 985583
SGI-Modid: xfs-linux-melb:xfs-kern:32183a
Signed-off-by: Christoph Hellwig
Signed-off-by: Lachlan McIlroy
Signed-off-by: Bill O'Donnell
Signed-off-by: David Chinner
