10 Jul, 2013
1 commit
-
Pull Ceph updates from Sage Weil:
"There is some follow-on RBD cleanup after the last window's code drop,
a series from Yan fixing multi-mds behavior in cephfs, and then a
sprinkling of bug fixes all around. Some warnings, sleeping while
atomic, a null dereference, and cleanups"* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client: (36 commits)
libceph: fix invalid unsigned->signed conversion for timespec encoding
libceph: call r_unsafe_callback when unsafe reply is received
ceph: fix race between cap issue and revoke
ceph: fix cap revoke race
ceph: fix pending vmtruncate race
ceph: avoid accessing invalid memory
libceph: Fix NULL pointer dereference in auth client code
ceph: Reconstruct the func ceph_reserve_caps.
ceph: Free mdsc if alloc mdsc->mdsmap failed.
ceph: remove sb_start/end_write in ceph_aio_write.
ceph: avoid meaningless calling ceph_caps_revoking if sync_mode == WB_SYNC_ALL.
ceph: fix sleeping function called from invalid context.
ceph: move inode to proper flushing list when auth MDS changes
rbd: fix a couple warnings
ceph: clear migrate seq when MDS restarts
ceph: check migrate seq before changing auth cap
ceph: fix race between page writeback and truncate
ceph: reset iov_len when discarding cap release messages
ceph: fix cap release race
libceph: fix truncate size calculation
...
05 Jul, 2013
1 commit
-
Signed-off-by: Al Viro
04 Jul, 2013
16 commits
-
If we receive new caps from the auth MDS and the non-auth MDS is
revoking the newly issued caps, we should release the caps from
the non-auth MDS. The scenario is filelock's state changes from
SYNC to LOCK. Non-auth MDS revokes Fc cap, the client gets Fc cap
from the auth MDS at the same time.Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
If caps are been revoking by the auth MDS, don't consider them as
issued even they are still issued by non-auth MDS. The non-auth
MDS should also be revoking/exporting these caps, the client just
hasn't received the cap revoke/export message.The race I encountered is: When caps are exporting to new MDS, the
client receives cap import message and cap revoke message from the
new MDS, then receives cap export message from the old MDS. When
the client receives cap revoke message from the new MDS, the revoking
caps are still issued by the old MDS, so the client does nothing.
Later when the cap export message is received, the client removes
the caps issued by the old MDS. (Another way to fix the race is
calling ceph_check_caps() in handle_cap_export())Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
The locking order for pending vmtruncate is wrong, it can lead to
following race:write wmtruncate work
------------------------ ----------------------
lock i_mutex
check i_truncate_pending check i_truncate_pending
truncate_inode_pages() lock i_mutex (blocked)
copy data to page cache
unlock i_mutex
truncate_inode_pages()The fix is take i_mutex before calling __ceph_do_pending_vmtruncate()
Fixes: http://tracker.ceph.com/issues/5453
Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
when mounting ceph with a dev name that starts with a slash, ceph
would attempt to access the character before that slash. Since we
don't actually own that byte of memory, we would trigger an
invalid access:[ 43.499934] BUG: unable to handle kernel paging request at ffff880fa3a97fff
[ 43.500984] IP: [] parse_mount_options+0x1a4/0x300
[ 43.501491] PGD 743b067 PUD 10283c4067 PMD 10282a6067 PTE 8000000fa3a97060
[ 43.502301] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 43.503006] Dumping ftrace buffer:
[ 43.503596] (ftrace buffer empty)
[ 43.504046] CPU: 0 PID: 10879 Comm: mount Tainted: G W 3.10.0-sasha #1129
[ 43.504851] task: ffff880fa625b000 ti: ffff880fa3412000 task.ti: ffff880fa3412000
[ 43.505608] RIP: 0010:[] [] parse_mount_options$
[ 43.506552] RSP: 0018:ffff880fa3413d08 EFLAGS: 00010286
[ 43.507133] RAX: ffff880fa3a98000 RBX: ffff880fa3a98000 RCX: 0000000000000000
[ 43.507893] RDX: ffff880fa3a98001 RSI: 000000000000002f RDI: ffff880fa3a98000
[ 43.508610] RBP: ffff880fa3413d58 R08: 0000000000001f99 R09: ffff880fa3fe64c0
[ 43.509426] R10: ffff880fa3413d98 R11: ffff880fa38710d8 R12: ffff880fa3413da0
[ 43.509792] R13: ffff880fa3a97fff R14: 0000000000000000 R15: ffff880fa3413d90
[ 43.509792] FS: 00007fa9c48757e0(0000) GS:ffff880fd2600000(0000) knlGS:000000000000$
[ 43.509792] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 43.509792] CR2: ffff880fa3a97fff CR3: 0000000fa3bb9000 CR4: 00000000000006b0
[ 43.509792] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 43.509792] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 43.509792] Stack:
[ 43.509792] 0000e5180000000e ffffffff85ca1900 ffff880fa38710d8 ffff880fa3413d98
[ 43.509792] 0000000000000120 0000000000000000 ffff880fa3a98000 0000000000000000
[ 43.509792] ffffffff85cf32a0 0000000000000000 ffff880fa3413dc8 ffffffff818f3c72
[ 43.509792] Call Trace:
[ 43.509792] [] ceph_mount+0xa2/0x390
[ 43.509792] [] ? pcpu_alloc+0x334/0x3c0
[ 43.509792] [] mount_fs+0x8d/0x1a0
[ 43.509792] [] ? __alloc_percpu+0x10/0x20
[ 43.509792] [] vfs_kern_mount+0x79/0x100
[ 43.509792] [] do_new_mount+0xcd/0x1c0
[ 43.509792] [] do_mount+0x15d/0x210
[ 43.509792] [] ? strndup_user+0x45/0x60
[ 43.509792] [] SyS_mount+0x9d/0xe0
[ 43.509792] [] tracesys+0xdd/0xe2
[ 43.509792] Code: 4c 8b 5d c0 74 0a 48 8d 50 01 49 89 14 24 eb 17 31 c0 48 83 c9 ff $
[ 43.509792] RIP [] parse_mount_options+0x1a4/0x300
[ 43.509792] RSP
[ 43.509792] CR2: ffff880fa3a97fff
[ 43.509792] ---[ end trace 22469cd81e93af51 ]---Signed-off-by: Sasha Levin
Reviewed-by: Sage Weil -
Drop ignored return value. Fix allocation failure case to not leak.
Signed-off-by: Jianpeng Ma
Reviewed-by: Sage Weil -
Signed-off-by: Jianpeng Ma
Reviewed-by: Sage Weil -
Either in vfs_write or io_submit,it call file_start/end_write.
The different between file_start/end_write and sb_start/end_write is
file_ only handle regular file.But i think in ceph_aio_write,it only
for regular file.Signed-off-by: Jianpeng Ma
Acked-by: Yan, Zheng -
Signed-off-by: Jianpeng Ma
Reviewed-by: Sage Weil -
[ 1121.231883] BUG: sleeping function called from invalid context at kernel/rwsem.c:20
[ 1121.231935] in_atomic(): 1, irqs_disabled(): 0, pid: 9831, name: mv
[ 1121.231971] 1 lock held by mv/9831:
[ 1121.231973] #0: (&(&ci->i_ceph_lock)->rlock){+.+...},at:[] ceph_getxattr+0x58/0x1d0 [ceph]
[ 1121.231998] CPU: 3 PID: 9831 Comm: mv Not tainted 3.10.0-rc6+ #215
[ 1121.232000] Hardware name: To Be Filled By O.E.M. To Be Filled By
O.E.M./To be filled by O.E.M., BIOS 080015 11/09/2011
[ 1121.232027] ffff88006d355a80 ffff880092f69ce0 ffffffff8168348c ffff880092f69cf8
[ 1121.232045] ffffffff81070435 ffff88006d355a20 ffff880092f69d20 ffffffff816899ba
[ 1121.232052] 0000000300000004 ffff8800b76911d0 ffff88006d355a20 ffff880092f69d68
[ 1121.232056] Call Trace:
[ 1121.232062] [] dump_stack+0x19/0x1b
[ 1121.232067] [] __might_sleep+0xe5/0x110
[ 1121.232071] [] down_read+0x2a/0x98
[ 1121.232080] [] ceph_vxattrcb_layout+0x60/0xf0 [ceph]
[ 1121.232088] [] ceph_getxattr+0x9f/0x1d0 [ceph]
[ 1121.232093] [] vfs_getxattr+0xa8/0xd0
[ 1121.232097] [] getxattr+0xab/0x1c0
[ 1121.232100] [] ? final_putname+0x22/0x50
[ 1121.232104] [] ? kmem_cache_free+0xb0/0x260
[ 1121.232107] [] ? final_putname+0x22/0x50
[ 1121.232110] [] ? trace_hardirqs_on+0xd/0x10
[ 1121.232114] [] ? sysret_check+0x1b/0x56
[ 1121.232120] [] SyS_fgetxattr+0x6c/0xc0
[ 1121.232125] [] system_call_fastpath+0x16/0x1b
[ 1121.232129] BUG: scheduling while atomic: mv/9831/0x10000002
[ 1121.232154] 1 lock held by mv/9831:
[ 1121.232156] #0: (&(&ci->i_ceph_lock)->rlock){+.+...}, at:
[] ceph_getxattr+0x58/0x1d0 [ceph]I think move the ci->i_ceph_lock down is safe because we can't free
ceph_inode_info at there.CC: stable@vger.kernel.org # 3.8+
Signed-off-by: Jianpeng Ma
Reviewed-by: Sage Weil -
Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
We may receive old request reply from the exporter MDS after receiving
the importer MDS' cap import message.Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
The client can receive truncate request from MDS at any time.
So the page writeback code need to get i_size, truncate_seq and
truncate_size atomicallySigned-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
ceph_encode_inode_release() can race with ceph_open() and release
caps wanted by open files. So it should call __ceph_caps_wanted()
to get the wanted caps.Signed-off-by: Yan, Zheng
Reviewed-by: Sage Weil -
Pull second set of VFS changes from Al Viro:
"Assorted f_pos race fixes, making do_splice_direct() safe to call with
i_mutex on parent, O_TMPFILE support, Jeff's locks.c series,
->d_hash/->d_compare calling conventions changes from Linus, misc
stuff all over the place."* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (63 commits)
Document ->tmpfile()
ext4: ->tmpfile() support
vfs: export lseek_execute() to modules
lseek_execute() doesn't need an inode passed to it
block_dev: switch to fixed_size_llseek()
cpqphp_sysfs: switch to fixed_size_llseek()
tile-srom: switch to fixed_size_llseek()
proc_powerpc: switch to fixed_size_llseek()
ubi/cdev: switch to fixed_size_llseek()
pci/proc: switch to fixed_size_llseek()
isapnp: switch to fixed_size_llseek()
lpfc: switch to fixed_size_llseek()
locks: give the blocked_hash its own spinlock
locks: add a new "lm_owner_key" lock operation
locks: turn the blocked_list into a hashtable
locks: convert fl_link to a hlist_node
locks: avoid taking global lock if possible when waking up blocked waiters
locks: protect most of the file_lock handling with i_lock
locks: encapsulate the fl_link list handling
locks: make "added" in __posix_lock_file a bool
...
03 Jul, 2013
2 commits
-
For those file systems(btrfs/ext4/ocfs2/tmpfs) that support
SEEK_DATA/SEEK_HOLE functions, we end up handling the similar
matter in lseek_execute() to update the current file offset
to the desired offset if it is valid, ceph also does the
simliar things at ceph_llseek().To reduce the duplications, this patch make lseek_execute()
public accessible so that we can call it directly from the
underlying file systems.Thanks Dave Chinner for this suggestion.
[AV: call it vfs_setpos(), don't bring the removed 'inode' argument back]
v2->v1:
- Add kernel-doc comments for lseek_execute()
- Call lseek_execute() in ceph->llseek()Signed-off-by: Jie Liu
Cc: Dave Chinner
Cc: Al Viro
Cc: Andi Kleen
Cc: Andrew Morton
Cc: Christoph Hellwig
Cc: Chris Mason
Cc: Josef Bacik
Cc: Ben Myers
Cc: Ted Tso
Cc: Hugh Dickins
Cc: Mark Fasheh
Cc: Joel Becker
Cc: Sage Weil
Signed-off-by: Al Viro -
Pull ext4 update from Ted Ts'o:
"Lots of bug fixes, cleanups and optimizations. In the bug fixes
category, of note is a fix for on-line resizing file systems where the
block size is smaller than the page size (i.e., file systems 1k blocks
on x86, or more interestingly file systems with 4k blocks on Power or
ia64 systems.)In the cleanup category, the ext4's punch hole implementation was
significantly improved by Lukas Czerner, and now supports bigalloc
file systems. In addition, Jan Kara significantly cleaned up the
write submission code path. We also improved error checking and added
a few sanity checks.In the optimizations category, two major optimizations deserve
mention. The first is that ext4_writepages() is now used for
nodelalloc and ext3 compatibility mode. This allows writes to be
submitted much more efficiently as a single bio request, instead of
being sent as individual 4k writes into the block layer (which then
relied on the elevator code to coalesce the requests in the block
queue). Secondly, the extent cache shrink mechanism, which was
introduce in 3.9, no longer has a scalability bottleneck caused by the
i_es_lru spinlock. Other optimizations include some changes to reduce
CPU usage and to avoid issuing empty commits unnecessarily."* tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4: (86 commits)
ext4: optimize starting extent in ext4_ext_rm_leaf()
jbd2: invalidate handle if jbd2_journal_restart() fails
ext4: translate flag bits to strings in tracepoints
ext4: fix up error handling for mpage_map_and_submit_extent()
jbd2: fix theoretical race in jbd2__journal_restart
ext4: only zero partial blocks in ext4_zero_partial_blocks()
ext4: check error return from ext4_write_inline_data_end()
ext4: delete unnecessary C statements
ext3,ext4: don't mess with dir_file->f_pos in htree_dirblock_to_tree()
jbd2: move superblock checksum calculation to jbd2_write_superblock()
ext4: pass inode pointer instead of file pointer to punch hole
ext4: improve free space calculation for inline_data
ext4: reduce object size when !CONFIG_PRINTK
ext4: improve extent cache shrink mechanism to avoid to burn CPU time
ext4: implement error handling of ext4_mb_new_preallocation()
ext4: fix corruption when online resizing a fs with 1K block size
ext4: delete unused variables
ext4: return FIEMAP_EXTENT_UNKNOWN for delalloc extents
jbd2: remove debug dependency on debug_fs and update Kconfig help text
jbd2: use a single printk for jbd_debug()
...
02 Jul, 2013
3 commits
-
I introduced a new temporary variable "info" instead of
"m->m_info[mds]". Also I reversed the if condition and pulled
everything in one indent level.Signed-off-by: Dan Carpenter
Reviewed-by: Alex Elder -
This patch makes the following improvements to the error handling
in the ceph_mdsmap_decode function:- Add a NULL check for return value from kcalloc
- Make use of the variable errSigned-off-by: Emil Goode
Signed-off-by: Sage Weil -
Signed-off-by: Jim Schutt
Reviewed-by: Alex Elder
29 Jun, 2013
2 commits
-
Having a global lock that protects all of this code is a clear
scalability problem. Instead of doing that, move most of the code to be
protected by the i_lock instead. The exceptions are the global lists
that the ->fl_link sits on, and the ->fl_block list.->fl_link is what connects these structures to the
global lists, so we must ensure that we hold those locks when iterating
over or updating these lists.Furthermore, sound deadlock detection requires that we hold the
blocked_list state steady while checking for loops. We also must ensure
that the search and update to the list are atomic.For the checking and insertion side of the blocked_list, push the
acquisition of the global lock into __posix_lock_file and ensure that
checking and update of the blocked_list is done without dropping the
lock in between.On the removal side, when waking up blocked lock waiters, take the
global lock before walking the blocked list and dequeue the waiters from
the global list prior to removal from the fl_block list.With this, deadlock detection should be race free while we minimize
excessive file_lock_lock thrashing.Finally, in order to avoid a lock inversion problem when handling
/proc/locks output we must ensure that manipulations of the fl_block
list are also protected by the file_lock_lock.Signed-off-by: Jeff Layton
Signed-off-by: Al Viro -
Signed-off-by: Al Viro
12 Jun, 2013
1 commit
-
Pull ceph fixes from Sage Weil:
"There is a pair of fixes for double-frees in the recent bundle for
3.10, a couple of fixes for long-standing bugs (sleep while atomic and
an endianness fix), and a locking fix that can be triggered when osds
are going down"* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client:
rbd: fix cleanup in rbd_add()
rbd: don't destroy ceph_opts in rbd_add()
ceph: ceph_pagelist_append might sleep while atomic
ceph: add cpu_to_le32() calls when encoding a reconnect capability
libceph: must hold mutex for reset_changed_osds()
22 May, 2013
2 commits
-
->invalidatepage() aop now accepts range to invalidate so we can make
use of it in ceph_invalidatepage().Signed-off-by: Lukas Czerner
Acked-by: Sage Weil
Cc: ceph-devel@vger.kernel.org -
Currently there is no way to truncate partial page where the end
truncate point is not at the end of the page. This is because it was not
needed and the functionality was enough for file system truncate
operation to work properly. However more file systems now support punch
hole feature and it can benefit from mm supporting truncating page just
up to the certain point.Specifically, with this functionality truncate_inode_pages_range() can
be changed so it supports truncating partial page at the end of the
range (currently it will BUG_ON() if 'end' is not at the end of the
page).This commit changes the invalidatepage() address space operation
prototype to accept range to be invalidated and update all the instances
for it.We also change the block_invalidatepage() in the same way and actually
make a use of the new length argument implementing range invalidation.Actual file system implementations will follow except the file systems
where the changes are really simple and should not change the behaviour
in any way .Implementation for truncate_page_range() which will be able
to accept page unaligned ranges will follow as well.Signed-off-by: Lukas Czerner
Cc: Andrew Morton
Cc: Hugh Dickins
18 May, 2013
2 commits
-
Ceph's encode_caps_cb() worked hard to not call __page_cache_alloc()
while holding a lock, but it's spoiled because ceph_pagelist_addpage()
always calls kmap(), which might sleep. Here's the result:[13439.295457] ceph: mds0 reconnect start
[13439.300572] BUG: sleeping function called from invalid context at include/linux/highmem.h:58
[13439.309243] in_atomic(): 1, irqs_disabled(): 0, pid: 12059, name: kworker/1:1
. . .
[13439.376225] Call Trace:
[13439.378757] [] __might_sleep+0xfc/0x110
[13439.384353] [] ceph_pagelist_append+0x120/0x1b0 [libceph]
[13439.391491] [] ceph_encode_locks+0x89/0x190 [ceph]
[13439.398035] [] ? _raw_spin_lock+0x49/0x50
[13439.403775] [] ? lock_flocks+0x15/0x20
[13439.409277] [] encode_caps_cb+0x41f/0x4a0 [ceph]
[13439.415622] [] ? igrab+0x28/0x70
[13439.420610] [] ? iterate_session_caps+0xe8/0x250 [ceph]
[13439.427584] [] iterate_session_caps+0x115/0x250 [ceph]
[13439.434499] [] ? set_request_path_attr+0x2d0/0x2d0 [ceph]
[13439.441646] [] send_mds_reconnect+0x238/0x450 [ceph]
[13439.448363] [] ? ceph_mdsmap_decode+0x5e2/0x770 [ceph]
[13439.455250] [] check_new_map+0x352/0x500 [ceph]
[13439.461534] [] ceph_mdsc_handle_map+0x1bd/0x260 [ceph]
[13439.468432] [] ? mutex_unlock+0xe/0x10
[13439.473934] [] extra_mon_dispatch+0x22/0x30 [ceph]
[13439.480464] [] dispatch+0xbc/0x110 [libceph]
[13439.486492] [] process_message+0x1ad/0x1d0 [libceph]
[13439.493190] [] ? read_partial_message+0x3e8/0x520 [libceph]
. . .
[13439.587132] ceph: mds0 reconnect success
[13490.720032] ceph: mds0 caps stale
[13501.235257] ceph: mds0 recovery completed
[13501.300419] ceph: mds0 caps renewedFix it up by encoding locks into a buffer first, and when the number
of encoded locks is stable, copy that into a ceph_pagelist.[elder@inktank.com: abbreviated the stack info a bit.]
Cc: stable@vger.kernel.org # 3.4+
Signed-off-by: Jim Schutt
Reviewed-by: Alex Elder -
In his review, Alex Elder mentioned that he hadn't checked that
num_fcntl_locks and num_flock_locks were properly decoded on the
server side, from a le32 over-the-wire type to a cpu type.
I checked, and AFAICS it is done; those interested can consult
Locker::_do_cap_update()
in src/mds/Locker.cc and src/include/encoding.h in the Ceph server
code (git://github.com/ceph/ceph).I also checked the server side for flock_len decoding, and I believe
that also happens correctly, by virtue of having been declared
__le32 in struct ceph_mds_cap_reconnect, in src/include/ceph_fs.h.Cc: stable@vger.kernel.org # 3.4+
Signed-off-by: Jim Schutt
Reviewed-by: Alex Elder
08 May, 2013
1 commit
-
Faster kernel compiles by way of fewer unnecessary includes.
[akpm@linux-foundation.org: fix fallout]
[akpm@linux-foundation.org: fix build]
Signed-off-by: Kent Overstreet
Cc: Zach Brown
Cc: Felipe Balbi
Cc: Greg Kroah-Hartman
Cc: Mark Fasheh
Cc: Joel Becker
Cc: Rusty Russell
Cc: Jens Axboe
Cc: Asai Thambi S P
Cc: Selvan Mani
Cc: Sam Bradshaw
Cc: Jeff Moyer
Cc: Al Viro
Cc: Benjamin LaHaise
Reviewed-by: "Theodore Ts'o"
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
02 May, 2013
9 commits
-
Now that we have a library routine to create snap contexts, use it.
This is part of:
http://tracker.ceph.com/issues/4857Signed-off-by: Alex Elder
Reviewed-by: Josh Durgin -
In the incremental move toward supporting distinct data items in an
osd request some of the functions had "write_request" parameters to
indicate, basically, whether the data belonged to in_data or the
out_data. Now that we maintain the data fields in the op structure
there is no need to indicate the direction, so get rid of the
"write_request" parameters.Signed-off-by: Alex Elder
Reviewed-by: Josh Durgin -
Fix printk format warnings by using %zd for 'ssize_t' variables:
fs/ceph/file.c:751:2: warning: format '%ld' expects argument of type 'long int', but argument 11 has type 'ssize_t' [-Wformat]
fs/ceph/file.c:762:2: warning: format '%ld' expects argument of type 'long int', but argument 11 has type 'ssize_t' [-Wformat]Signed-off-by: Randy Dunlap
Cc: ceph-devel@vger.kernel.org
Signed-off-by: Sage Weil -
ceph_writepages_start() reads inode->i_size in two places. It can get
different values between successive read, because truncate can change
inode->i_size at any time. The race can lead to mismatch between data
length of osd request and pages marked as writeback. When osd request
finishes, it clear writeback page according to its data length. So
some pages can be left in writeback state forever. The fix is only
read inode->i_size once, save its value to a local variable and use
the local variable when i_size is needed.Signed-off-by: Yan, Zheng
Reviewed-by: Alex Elder -
copy write checks in __generic_file_aio_write to ceph_aio_write.
To make these checks cover sync write path.Signed-off-by: Yan, Zheng
Reviewed-by: Alex Elder -
There is deadlock as illustrated bellow. The fix is taking i_mutex
before getting Fw cap reference.write truncate MDS
--------------------- -------------------- --------------
get Fw cap
lock i_mutex
lock i_mutex (blocked)
request setattr.size ->
Reviewed-by: Alex Elder
Reviewed-by: Sage Weil -
An osd request currently has two callbacks. They inform the
initiator of the request when we've received confirmation for the
target osd that a request was received, and when the osd indicates
all changes described by the request are durable.The only time the second callback is used is in the ceph file system
for a synchronous write. There's a race that makes some handling of
this case unsafe. This patch addresses this problem. The error
handling for this callback is also kind of gross, and this patch
changes that as well.In ceph_sync_write(), if a safe callback is requested we want to add
the request on the ceph inode's unsafe items list. Because items on
this list must have their tid set (by ceph_osd_start_request()), the
request added *after* the call to that function returns. The
problem with this is that there's a race between starting the
request and adding it to the unsafe items list; the request may
already be complete before ceph_sync_write() even begins to put it
on the list.To address this, we change the way the "safe" callback is used.
Rather than just calling it when the request is "safe", we use it to
notify the initiator the bounds (start and end) of the period during
which the request is *unsafe*. So the initiator gets notified just
before the request gets sent to the osd (when it is "unsafe"), and
again when it's known the results are durable (it's no longer
unsafe). The first call will get made in __send_request(), just
before the request message gets sent to the messenger for the first
time. That function is only called by __send_queued(), which is
always called with the osd client's request mutex held.We then have this callback function insert the request on the ceph
inode's unsafe list when we're told the request is unsafe. This
will avoid the race because this call will be made under protection
of the osd client's request mutex. It also nicely groups the setup
and cleanup of the state associated with managing unsafe requests.The name of the "safe" callback field is changed to "unsafe" to
better reflect its new purpose. It has a Boolean "unsafe" parameter
to indicate whether the request is becoming unsafe or is now safe.
Because the "msg" parameter wasn't used, we drop that.This resolves the original problem reportedin:
http://tracker.ceph.com/issues/4706Reported-by: Yan, Zheng
Signed-off-by: Alex Elder
Reviewed-by: Yan, Zheng
Reviewed-by: Sage Weil -
In ceph_sync_write(), if a safe callback is supplied with a request,
and an error is returned by ceph_osdc_wait_request(), a block of
code is executed to remove the request from the unsafe writes list
and drop references to capabilities acquired just prior to a call to
ceph_osdc_wait_request().The only function used for this callback is sync_write_commit(),
and it does *exactly* what that block of error handling code does.Now in ceph_osdc_wait_request(), if an error occurs (due to an
interupt during a wait_for_completion_interruptible() call),
complete_request() gets called, and that calls the request's
safe_callback method if it's defined.So this means that this cleanup activity gets called twice in this
case, which is erroneous (and in fact leads to a crash).Fix this by just letting the osd client handle the cleanup in
the event of an interrupt.This resolves one problem mentioned in:
http://tracker.ceph.com/issues/4706Signed-off-by: Alex Elder
Reviewed-by: Yan, Zheng -
add getattr/setattr and xattrs related methods.
Signed-off-by: Yan, Zheng
Reviewed-by: Greg Farnum
