21 Dec, 2012
8 commits
-
Mark as cancelled an operation that is in progress rather than pending at the
time it is cancelled, and call fscache_complete_op() to cancel an operation so
that blocked ops can be started.Signed-off-by: David Howells
-
Don't mask off the object event mask when printing it. That way it can be seen
if threre are bits set that shouldn't be.Signed-off-by: David Howells
-
CacheFiles is missing some calls to fscache_retrieval_complete() in the error
handling/collision paths of its reader functions.This can be seen by the following assertion tripping in fscache_put_operation()
whereby the operation being destroyed is still in the in-progress state and has
not been cancelled or completed:FS-Cache: Assertion failed
3 == 5 is false
------------[ cut here ]------------
kernel BUG at fs/fscache/operation.c:408!
invalid opcode: 0000 [#1] SMP
CPU 2
Modules linked in: xfs ioatdma dca loop joydev evdev
psmouse dcdbas pcspkr serio_raw i5000_edac edac_core i5k_amb shpchp
pci_hotplug sg sr_mod]Pid: 8062, comm: httpd Not tainted 3.1.0-rc8 #1 Dell Inc. PowerEdge 1950/0DT097
RIP: 0010:[] [] fscache_put_operation+0x304/0x330
RSP: 0018:ffff880062f739d8 EFLAGS: 00010296
RAX: 0000000000000025 RBX: ffff8800c5122e84 RCX: ffffffff81ddf040
RDX: 00000000ffffffff RSI: 0000000000000082 RDI: ffffffff81ddef30
RBP: ffff880062f739f8 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000003 R12: ffff8800c5122e40
R13: ffff880037a2cd20 R14: ffff880087c7a058 R15: ffff880087c7a000
FS: 00007f63dcf636e0(0000) GS:ffff88022fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0c0a91f000 CR3: 0000000062ec2000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process httpd (pid: 8062, threadinfo ffff880062f72000, task ffff880087e58000)
Stack:
ffff880062f73bf8 0000000000000000 ffff880062f73bf8 ffff880037a2cd20
ffff880062f73a68 ffffffff8119aa7e ffff88006540e000 ffff880062f73ad4
ffff88008e9a4308 ffff880037a2cd20 ffff880062f73a48 ffff8800c5122e40
Call Trace:
[] __fscache_read_or_alloc_pages+0x1fe/0x530
[] __nfs_readpages_from_fscache+0x70/0x1c0
[] nfs_readpages+0xca/0x1e0
[] ? rpc_do_put_task+0x36/0x50
[] ? alloc_nfs_open_context+0x4b/0x110
[] ? rpc_call_sync+0x5a/0x70
[] __do_page_cache_readahead+0x1ca/0x270
[] ra_submit+0x21/0x30
[] ondemand_readahead+0x11d/0x250
[] page_cache_sync_readahead+0x36/0x60
[] generic_file_aio_read+0x454/0x770
[] nfs_file_read+0xe1/0x130
[] do_sync_read+0xd9/0x120
[] ? mntput+0x1f/0x40
[] ? fput+0x1cb/0x260
[] vfs_read+0xc8/0x180
[] sys_read+0x55/0x90Reported-by: Mark Moseley
Signed-off-by: David Howells -
Implement invalidation for CacheFiles. This is in two parts:
(1) Provide an invalidation method (which just truncates the backing file).
(2) Abort attempts to copy anything read from the backing file whilst
invalidation is in progress.Question: CacheFiles uses truncation in a couple of places. It has been using
notify_change() rather than sys_truncate() or something similar. This means
it bypasses a bunch of checks and suchlike that it possibly should be making
(security, file locking, lease breaking, vfsmount write). Should it be using
vfs_truncate() as added by a preceding patch or should it use notify_write()
and assume that anyone poking around in the cache files on disk gets
everything they deserve?Signed-off-by: David Howells
-
Fix the state management of internal fscache operations and the accounting of
what operations are in what states.This is done by:
(1) Give struct fscache_operation a enum variable that directly represents the
state it's currently in, rather than spreading this knowledge over a bunch
of flags, who's processing the operation at the moment and whether it is
queued or not.This makes it easier to write assertions to check the state at various
points and to prevent invalid state transitions.(2) Add an 'operation complete' state and supply a function to indicate the
completion of an operation (fscache_op_complete()) and make things call
it. The final call to fscache_put_operation() can then check that an op
in the appropriate state (complete or cancelled).(3) Adjust the use of object->n_ops, ->n_in_progress, ->n_exclusive to better
govern the state of an object:(a) The ->n_ops is now the number of extant operations on the object
and is now decremented by fscache_put_operation() only.(b) The ->n_in_progress is simply the number of objects that have been
taken off of the object's pending queue for the purposes of being
run. This is decremented by fscache_op_complete() only.(c) The ->n_exclusive is the number of exclusive ops that have been
submitted and queued or are in progress. It is decremented by
fscache_op_complete() and by fscache_cancel_op().fscache_put_operation() and fscache_operation_gc() now no longer try to
clean up ->n_exclusive and ->n_in_progress. That was leading to double
decrements against fscache_cancel_op().fscache_cancel_op() now no longer decrements ->n_ops. That was leading to
double decrements against fscache_put_operation().fscache_submit_exclusive_op() now decides whether it has to queue an op
based on ->n_in_progress being > 0 rather than ->n_ops > 0 as the latter
will persist in being true even after all preceding operations have been
cancelled or completed. Furthermore, if an object is active and there are
runnable ops against it, there must be at least one op running.(4) Add a remaining-pages counter (n_pages) to struct fscache_retrieval and
provide a function to record completion of the pages as they complete.When n_pages reaches 0, the operation is deemed to be complete and
fscache_op_complete() is called.Add calls to fscache_retrieval_complete() anywhere we've finished with a
page we've been given to read or allocate for. This includes places where
we just return pages to the netfs for reading from the server and where
accessing the cache fails and we discard the proposed netfs page.The bugs in the unfixed state management manifest themselves as oopses like the
following where the operation completion gets out of sync with return of the
cookie by the netfs. This is possible because the cache unlocks and returns
all the netfs pages before recording its completion - which means that there's
nothing to stop the netfs discarding them and returning the cookie.FS-Cache: Cookie 'NFS.fh' still has outstanding reads
------------[ cut here ]------------
kernel BUG at fs/fscache/cookie.c:519!
invalid opcode: 0000 [#1] SMP
CPU 1
Modules linked in: cachefiles nfs fscache auth_rpcgss nfs_acl lockd sunrpcPid: 400, comm: kswapd0 Not tainted 3.1.0-rc7-fsdevel+ #1090 /DG965RY
RIP: 0010:[] [] __fscache_relinquish_cookie+0x170/0x343 [fscache]
RSP: 0018:ffff8800368cfb00 EFLAGS: 00010282
RAX: 000000000000003c RBX: ffff880023cc8790 RCX: 0000000000000000
RDX: 0000000000002f2e RSI: 0000000000000001 RDI: ffffffff813ab86c
RBP: ffff8800368cfb50 R08: 0000000000000002 R09: 0000000000000000
R10: ffff88003a1b7890 R11: ffff88001df6e488 R12: ffff880023d8ed98
R13: ffff880023cc8798 R14: 0000000000000004 R15: ffff88003b8bf370
FS: 0000000000000000(0000) GS:ffff88003bd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000008ba008 CR3: 0000000023d93000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process kswapd0 (pid: 400, threadinfo ffff8800368ce000, task ffff88003b8bf040)
Stack:
ffff88003b8bf040 ffff88001df6e528 ffff88001df6e528 ffffffffa00b46b0
ffff88003b8bf040 ffff88001df6e488 ffff88001df6e620 ffffffffa00b46b0
ffff88001ebd04c8 0000000000000004 ffff8800368cfb70 ffffffffa00b2c91
Call Trace:
[] nfs_fscache_release_inode_cookie+0x3b/0x47 [nfs]
[] nfs_clear_inode+0x3c/0x41 [nfs]
[] nfs4_evict_inode+0x2f/0x33 [nfs]
[] evict+0xa1/0x15c
[] dispose_list+0x2c/0x38
[] prune_icache_sb+0x28c/0x29b
[] prune_super+0xd5/0x140
[] shrink_slab+0x102/0x1ab
[] balance_pgdat+0x2f2/0x595
[] ? process_timeout+0xb/0xb
[] kswapd+0x270/0x289
[] ? __init_waitqueue_head+0x46/0x46
[] ? balance_pgdat+0x595/0x595
[] kthread+0x7f/0x87
[] kernel_thread_helper+0x4/0x10
[] ? finish_task_switch+0x45/0xc0
[] ? retint_restore_args+0xe/0xe
[] ? __init_kthread_worker+0x53/0x53
[] ? gs_change+0xb/0xbSigned-off-by: David Howells
-
Downgrade some debugging statements to not unconditionally print stuff, but
rather be conditional on the appropriate module parameter setting.Signed-off-by: David Howells
-
Downgrade the requirements passed to the allocator in the gfp flags parameter.
FS-Cache/CacheFiles can handle OOM conditions simply by aborting the attempt to
store an object or a page in the cache.Signed-off-by: David Howells
-
Under some circumstances CacheFiles defers the marking of pages with PG_fscache
so that it can take advantage of pagevecs to reduce the number of calls to
fscache_mark_pages_cached() and the netfs's hook to keep track of this.There are, however, two problems with this:
(1) It can lead to the PG_fscache mark being applied _after_ the page is set
PG_uptodate and unlocked (by the call to fscache_end_io()).(2) CacheFiles's ref on the page is dropped immediately following
fscache_end_io() - and so may not still be held when the mark is applied.
This can lead to the page being passed back to the allocator before the
mark is applied.Fix this by, where appropriate, marking the page before calling
fscache_end_io() and releasing the page. This means that we can't take
advantage of pagevecs and have to make a separate call for each page to the
marking routines.The symptoms of this are Bad Page state errors cropping up under memory
pressure, for example:BUG: Bad page state in process tar pfn:002da
page:ffffea0000009fb0 count:0 mapcount:0 mapping: (null) index:0x1447
page flags: 0x1000(private_2)
Pid: 4574, comm: tar Tainted: G W 3.1.0-rc4-fsdevel+ #1064
Call Trace:
[] ? dump_page+0xb9/0xbe
[] bad_page+0xd5/0xea
[] get_page_from_freelist+0x35b/0x46a
[] __alloc_pages_nodemask+0x362/0x662
[] __do_page_cache_readahead+0x13a/0x267
[] ? __do_page_cache_readahead+0xa2/0x267
[] ra_submit+0x1c/0x20
[] ondemand_readahead+0x28b/0x29a
[] ? ondemand_readahead+0x163/0x29a
[] page_cache_sync_readahead+0x38/0x3a
[] generic_file_aio_read+0x2ab/0x67e
[] nfs_file_read+0xa4/0xc9 [nfs]
[] do_sync_read+0xba/0xfa
[] ? security_file_permission+0x7b/0x84
[] ? rw_verify_area+0xab/0xc8
[] vfs_read+0xaa/0x13a
[] sys_read+0x45/0x6c
[] system_call_fastpath+0x16/0x1bAs can be seen, PG_private_2 (== PG_fscache) is set in the page flags.
Instrumenting fscache_mark_pages_cached() to verify whether page->mapping was
set appropriately showed that sometimes it wasn't. This led to the discovery
that sometimes the page has apparently been reclaimed by the time the marker
got to see it.Reported-by: M. Stevens
Signed-off-by: David Howells
Reviewed-by: Jeff Layton
31 Jul, 2012
1 commit
-
Support the caching of large files.
Addresses https://bugzilla.kernel.org/show_bug.cgi?id=31182
Signed-off-by: Justin Lecher
Signed-off-by: Suresh Jayaraman
Tested-by: Suresh Jayaraman
Acked-by: David Howells
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
23 Jul, 2012
1 commit
-
Signed-off-by: Al Viro
14 Jul, 2012
1 commit
-
all we want is a boolean flag, same as the method gets now
Signed-off-by: Al Viro
21 Mar, 2012
1 commit
-
Signed-off-by: Al Viro
04 Jan, 2012
1 commit
-
Move invalidate_bdev, block_sync_page into fs/block_dev.c. Export
kill_bdev as well, so brd doesn't have to open code it. Reduce
buffer_head.h requirement accordingly.Removed a rather large comment from invalidate_bdev, as it looked a bit
obsolete to bother moving. The small comment replacing it says enough.Signed-off-by: Nick Piggin
Cc: Al Viro
Cc: Christoph Hellwig
Signed-off-by: Andrew Morton
Signed-off-by: Al Viro
20 Jul, 2011
2 commits
-
never is...
Signed-off-by: Al Viro
-
it never is...
Signed-off-by: Al Viro
31 Mar, 2011
1 commit
-
Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: Lucas De Marchi
24 Jan, 2011
1 commit
-
Add calls to path-based security hooks into CacheFiles as, unlike inode-based
security, these aren't implicit in the vfs_mkdir() and similar calls.Reported-by: Tetsuo Handa
Signed-off-by: David Howells
Signed-off-by: James Morris
15 Oct, 2010
1 commit
-
All file_operations should get a .llseek operation so we can make
nonseekable_open the default for future file operations without a
.llseek pointer.The three cases that we can automatically detect are no_llseek, seq_lseek
and default_llseek. For cases where we can we can automatically prove that
the file offset is always ignored, we use noop_llseek, which maintains
the current behavior of not returning an error from a seek.New drivers should normally not use noop_llseek but instead use no_llseek
and call nonseekable_open at open time. Existing drivers can be converted
to do the same when the maintainer knows for certain that no user code
relies on calling seek on the device file.The generated code is often incorrectly indented and right now contains
comments that clarify for each added line why a specific variant was
chosen. In the version that gets submitted upstream, the comments will
be gone and I will manually fix the indentation, because there does not
seem to be a way to do that using coccinelle.Some amount of new code is currently sitting in linux-next that should get
the same modifications, which I will do at the end of the merge window.Many thanks to Julia Lawall for helping me learn to write a semantic
patch that does all this.===== begin semantic patch =====
// This adds an llseek= method to all file operations,
// as a preparation for making no_llseek the default.
//
// The rules are
// - use no_llseek explicitly if we do nonseekable_open
// - use seq_lseek for sequential files
// - use default_llseek if we know we access f_pos
// - use noop_llseek if we know we don't access f_pos,
// but we still want to allow users to call lseek
//
@ open1 exists @
identifier nested_open;
@@
nested_open(...)
{}
@ open exists@
identifier open_f;
identifier i, f;
identifier open1.nested_open;
@@
int open_f(struct inode *i, struct file *f)
{}
@ read disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{}
@ read_no_fpos disable optional_qualifier exists @
identifier read_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
{
... when != off
}@ write @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
expression E;
identifier func;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{}
@ write_no_fpos @
identifier write_f;
identifier f, p, s, off;
type ssize_t, size_t, loff_t;
@@
ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
{
... when != off
}@ fops0 @
identifier fops;
@@
struct file_operations fops = {
...
};@ has_llseek depends on fops0 @
identifier fops0.fops;
identifier llseek_f;
@@
struct file_operations fops = {
...
.llseek = llseek_f,
...
};@ has_read depends on fops0 @
identifier fops0.fops;
identifier read_f;
@@
struct file_operations fops = {
...
.read = read_f,
...
};@ has_write depends on fops0 @
identifier fops0.fops;
identifier write_f;
@@
struct file_operations fops = {
...
.write = write_f,
...
};@ has_open depends on fops0 @
identifier fops0.fops;
identifier open_f;
@@
struct file_operations fops = {
...
.open = open_f,
...
};// use no_llseek if we call nonseekable_open
////////////////////////////////////////////
@ nonseekable1 depends on !has_llseek && has_open @
identifier fops0.fops;
identifier nso ~= "nonseekable_open";
@@
struct file_operations fops = {
... .open = nso, ...
+.llseek = no_llseek, /* nonseekable */
};@ nonseekable2 depends on !has_llseek @
identifier fops0.fops;
identifier open.open_f;
@@
struct file_operations fops = {
... .open = open_f, ...
+.llseek = no_llseek, /* open uses nonseekable */
};// use seq_lseek for sequential files
/////////////////////////////////////
@ seq depends on !has_llseek @
identifier fops0.fops;
identifier sr ~= "seq_read";
@@
struct file_operations fops = {
... .read = sr, ...
+.llseek = seq_lseek, /* we have seq_read */
};// use default_llseek if there is a readdir
///////////////////////////////////////////
@ fops1 depends on !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier readdir_e;
@@
// any other fop is used that changes pos
struct file_operations fops = {
... .readdir = readdir_e, ...
+.llseek = default_llseek, /* readdir is present */
};// use default_llseek if at least one of read/write touches f_pos
/////////////////////////////////////////////////////////////////
@ fops2 depends on !fops1 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read.read_f;
@@
// read fops use offset
struct file_operations fops = {
... .read = read_f, ...
+.llseek = default_llseek, /* read accesses f_pos */
};@ fops3 depends on !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write.write_f;
@@
// write fops use offset
struct file_operations fops = {
... .write = write_f, ...
+ .llseek = default_llseek, /* write accesses f_pos */
};// Use noop_llseek if neither read nor write accesses f_pos
///////////////////////////////////////////////////////////@ fops4 depends on !fops1 && !fops2 && !fops3 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
identifier write_no_fpos.write_f;
@@
// write fops use offset
struct file_operations fops = {
...
.write = write_f,
.read = read_f,
...
+.llseek = noop_llseek, /* read and write both use no f_pos */
};@ depends on has_write && !has_read && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier write_no_fpos.write_f;
@@
struct file_operations fops = {
... .write = write_f, ...
+.llseek = noop_llseek, /* write uses no f_pos */
};@ depends on has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
identifier read_no_fpos.read_f;
@@
struct file_operations fops = {
... .read = read_f, ...
+.llseek = noop_llseek, /* read uses no f_pos */
};@ depends on !has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
identifier fops0.fops;
@@
struct file_operations fops = {
...
+.llseek = noop_llseek, /* no read or write fn */
};
===== End semantic patch =====Signed-off-by: Arnd Bergmann
Cc: Julia Lawall
Cc: Christoph Hellwig
13 Aug, 2010
1 commit
-
Add a dummy printk function for the maintenance of unused printks through gcc
format checking, and also so that side-effect checking is maintained too.Signed-off-by: David Howells
Signed-off-by: Linus Torvalds
11 Aug, 2010
3 commits
-
Add three helpers that retrieve a refcounted copy of the root and cwd
from the supplied fs_struct.get_fs_root()
get_fs_pwd()
get_fs_root_and_pwd()Signed-off-by: Miklos Szeredi
Signed-off-by: Al Viro -
Dentry references should not be acquired without a corresponding
vfsmount ref.Signed-off-by: Miklos Szeredi
Acked-by: David Howells
Signed-off-by: Al Viro -
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6: (96 commits)
no need for list_for_each_entry_safe()/resetting with superblock list
Fix sget() race with failing mount
vfs: don't hold s_umount over close_bdev_exclusive() call
sysv: do not mark superblock dirty on remount
sysv: do not mark superblock dirty on mount
btrfs: remove junk sb_dirt change
BFS: clean up the superblock usage
AFFS: wait for sb synchronization when needed
AFFS: clean up dirty flag usage
cifs: truncate fallout
mbcache: fix shrinker function return value
mbcache: Remove unused features
add f_flags to struct statfs(64)
pass a struct path to vfs_statfs
update VFS documentation for method changes.
All filesystems that need invalidate_inode_buffers() are doing that explicitly
convert remaining ->clear_inode() to ->evict_inode()
Make ->drop_inode() just return whether inode needs to be dropped
fs/inode.c:clear_inode() is gone
fs/inode.c:evict() doesn't care about delete vs. non-delete paths now
...Fix up trivial conflicts in fs/nilfs2/super.c
10 Aug, 2010
1 commit
-
We'll need the path to implement the flags field for statvfs support.
We do have it available in all callers except:- ecryptfs_statfs. This one doesn't actually need vfs_statfs but just
needs to do a caller to the lower filesystem statfs method.
- sys_ustat. Add a non-exported statfs_by_dentry helper for it which
doesn't won't be able to fill out the flags field later on.In addition rename the helpers for statfs vs fstatfs to do_*statfs instead
of the misleading vfs prefix.Signed-off-by: Christoph Hellwig
Signed-off-by: Al Viro
23 Jul, 2010
2 commits
-
Make fscache operation to use only workqueue instead of combination of
workqueue and slow-work. FSCACHE_OP_SLOW is dropped and
FSCACHE_OP_FAST is renamed to FSCACHE_OP_ASYNC and uses newly added
fscache_op_wq workqueue to execute op->processor().
fscache_operation_init_slow() is dropped and fscache_operation_init()
now takes @processor argument directly.* Unbound workqueue is used.
* fscache_retrieval_work() is no longer necessary as OP_ASYNC now does
the equivalent thing.* sysctl fscache.operation_max_active added to control concurrency.
The default value is nr_cpus clamped between 2 and
WQ_UNBOUND_MAX_ACTIVE.* debugfs support is dropped for now. Tracing API based debug
facility is planned to be added.Signed-off-by: Tejun Heo
Acked-by: David Howells -
Make fscache object state transition callbacks use workqueue instead
of slow-work. New dedicated unbound CPU workqueue fscache_object_wq
is created. get/put callbacks are renamed and modified to take
@object and called directly from the enqueue wrapper and the work
function. While at it, make all open coded instances of get/put to
use fscache_get/put_object().* Unbound workqueue is used.
* work_busy() output is printed instead of slow-work flags in object
debugging outputs. They mean basically the same thing bit-for-bit.* sysctl fscache.object_max_active added to control concurrency. The
default value is nr_cpus clamped between 4 and
WQ_UNBOUND_MAX_ACTIVE.* slow_work_sleep_till_thread_needed() is replaced with fscache
private implementation fscache_object_sleep_till_congested() which
waits on fscache_object_wq congestion.* debugfs support is dropped for now. Tracing API based debug
facility is planned to be added.Signed-off-by: Tejun Heo
Acked-by: David Howells
13 May, 2010
1 commit
-
cachefiles_determine_cache_security() is expected to return with a
security override in place. However, if set_create_files_as() fails, we
fail to do this. In this case, we should just reinstate the security
override that was set by the caller.Furthermore, if set_create_files_as() fails, we should dispose of the
new credentials we were in the process of creating.Signed-off-by: David Howells
Signed-off-by: Linus Torvalds
12 May, 2010
1 commit
-
Fix an occasional EIO returned by a call to vfs_unlink():
[ 4868.465413] CacheFiles: I/O Error: Unlink failed
[ 4868.465444] FS-Cache: Cache cachefiles stopped due to I/O error
[ 4947.320011] CacheFiles: File cache on md3 unregistering
[ 4947.320041] FS-Cache: Withdrawing cache "mycache"
[ 5127.348683] FS-Cache: Cache "mycache" added (type cachefiles)
[ 5127.348716] CacheFiles: File cache on md3 registered
[ 7076.871081] CacheFiles: I/O Error: Unlink failed
[ 7076.871130] FS-Cache: Cache cachefiles stopped due to I/O error
[ 7116.780891] CacheFiles: File cache on md3 unregistering
[ 7116.780937] FS-Cache: Withdrawing cache "mycache"
[ 7296.813394] FS-Cache: Cache "mycache" added (type cachefiles)
[ 7296.813432] CacheFiles: File cache on md3 registeredWhat happens is this:
(1) A cached NFS file is seen to have become out of date, so NFS retires the
object and immediately acquires a new object with the same key.(2) Retirement of the old object is done asynchronously - so the lookup/create
to generate the new object may be done first.This can be a problem as the old object and the new object must exist at
the same point in the backing filesystem (i.e. they must have the same
pathname).(3) The lookup for the new object sees that a backing file already exists,
checks to see whether it is valid and sees that it isn't. It then deletes
that file and creates a new one on disk.(4) The retirement phase for the old file is then performed. It tries to
delete the dentry it has, but ext4_unlink() returns -EIO because the inode
attached to that dentry no longer matches the inode number associated with
the filename in the parent directory.The trace below shows this quite well.
[md5sum] ==> __fscache_relinquish_cookie(ffff88002d12fb58{NFS.fh,ffff88002ce62100},1)
[md5sum] ==> __fscache_acquire_cookie({NFS.server},{NFS.fh},ffff88002ce62100)NFS has retired the old cookie and asked for a new one.
[kslowd] ==> fscache_object_state_machine({OBJ52,OBJECT_ACTIVE,24})
[kslowd] OBJECT_DYING]
[kslowd] ==> fscache_object_state_machine({OBJ53,OBJECT_INIT,0})
[kslowd] OBJECT_LOOKING_UP]
[kslowd] ==> fscache_object_state_machine({OBJ52,OBJECT_DYING,24})
[kslowd] OBJECT_RECYCLING]The old object (OBJ52) is going through the terminal states to get rid of it,
whilst the new object - (OBJ53) - is coming into being.[kslowd] ==> fscache_object_state_machine({OBJ53,OBJECT_LOOKING_UP,0})
[kslowd] ==> cachefiles_walk_to_object({ffff88003029d8b8},OBJ53,@68,)
[kslowd] lookup '@68'
[kslowd] next -> ffff88002ce41bd0 positive
[kslowd] advance
[kslowd] lookup 'Es0g00og0_Nd_XCYe3BOzvXrsBLMlN6aw16M1htaA'
[kslowd] next -> ffff8800369faac8 positiveThe new object has looked up the subdir in which the file would be in (getting
dentry ffff88002ce41bd0) and then looked up the file itself (getting dentry
ffff8800369faac8).[kslowd] validate 'Es0g00og0_Nd_XCYe3BOzvXrsBLMlN6aw16M1htaA'
[kslowd] ==> cachefiles_bury_object(,'@68','Es0g00og0_Nd_XCYe3BOzvXrsBLMlN6aw16M1htaA')
[kslowd] remove ffff8800369faac8 from ffff88002ce41bd0
[kslowd] unlink stale object
[kslowd] inode does not match i_ino.[kslowd] OBJECT_DEAD]
[kslowd] ==> fscache_object_state_machine({OBJ53,OBJECT_AVAILABLE,0})
[kslowd] OBJECT_ACTIVE](Note that the above trace includes extra information beyond that produced by
the upstream code).The fix is to note when an object that is being retired has had its object
deleted preemptively by a replacement object that is being created, and to
skip the second removal attempt in such a case.Reported-by: Greg M
Reported-by: Mark Moseley
Reported-by: Romain DEGEZ
Signed-off-by: David Howells
Signed-off-by: Linus Torvalds
30 Mar, 2010
1 commit
-
…it slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
20 Feb, 2010
1 commit
-
cachefiles_delete_object() can race with rename. It gets the parent directory
of the object it's asked to delete, then locks it - but rename may have changed
the object's parent between the get and the completion of the lock.However, if such a circumstance is detected, we abandon our attempt to delete
the object - since it's no longer in the index key path, it won't be seen
again by lookups of that key. The assumption is that cachefilesd may have
culled it by renaming it to the graveyard for later destruction.Signed-off-by: David Howells
Signed-off-by: Al Viro
17 Dec, 2009
2 commits
-
* do ima_get_count() in __dentry_open()
* stop doing that in followups
* move ima_path_check() to right after nameidata_to_filp()
* don't bump counters on itSigned-off-by: Al Viro
-
Signed-off-by: Al Viro
16 Dec, 2009
1 commit
-
Makes use of skip_spaces() defined in lib/string.c for removing leading
spaces from strings all over the tree.It decreases lib.a code size by 47 bytes and reuses the function tree-wide:
text data bss dec hex filename
64688 584 592 65864 10148 (TOTALS-BEFORE)
64641 584 592 65817 10119 (TOTALS-AFTER)Also, while at it, if we see (*str && isspace(*str)), we can be sure to
remove the first condition (*str) as the second one (isspace(*str)) also
evaluates to 0 whenever *str == 0, making it redundant. In other words,
"a char equals zero is never a space".Julia Lawall tried the semantic patch (http://coccinelle.lip6.fr) below,
and found occurrences of this pattern on 3 more files:
drivers/leds/led-class.c
drivers/leds/ledtrig-timer.c
drivers/video/output.c@@
expression str;
@@( // ignore skip_spaces cases
while (*str && isspace(*str)) { \(str++;\|++str;\) }
|
- *str &&
isspace(*str)
)Signed-off-by: André Goddard Rosa
Cc: Julia Lawall
Cc: Martin Schwidefsky
Cc: Jeff Dike
Cc: Ingo Molnar
Cc: Thomas Gleixner
Cc: "H. Peter Anvin"
Cc: Richard Purdie
Cc: Neil Brown
Cc: Kyle McMartin
Cc: Henrique de Moraes Holschuh
Cc: David Howells
Cc:
Cc: Samuel Ortiz
Cc: Patrick McHardy
Cc: Takashi Iwai
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
01 Dec, 2009
1 commit
-
When IMA is active, using dentry_open without updating the
IMA counters will result in free/open imbalance errors when
fput is eventually called.Signed-off-by: Marc Dionne
Signed-off-by: David Howells
Signed-off-by: Linus Torvalds
20 Nov, 2009
7 commits
-
Don't log the CacheFiles lookup/create object routined failing with ENOBUFS as
under high memory load or high cache load they can do this quite a lot. This
error simply means that the requested object cannot be created on disk due to
lack of space, or due to failure of the backing filesystem to find sufficient
resources.Signed-off-by: David Howells
-
Catch an overly long wait for an old, dying active object when we want to
replace it with a new one. The probability is that all the slow-work threads
are hogged, and the delete can't get a look in.What we do instead is:
(1) if there's nothing in the slow work queue, we sleep until either the dying
object has finished dying or there is something in the slow work queue
behind which we can queue our object.(2) if there is something in the slow work queue, we return ETIMEDOUT to
fscache_lookup_object(), which then puts us back on the slow work queue,
presumably behind the deletion that we're blocked by. We are then
deferred for a while until we work our way back through the queue -
without blocking a slow-work thread unnecessarily.A backtrace similar to the following may appear in the log without this patch:
INFO: task kslowd004:5711 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kslowd004 D 0000000000000000 0 5711 2 0x00000080
ffff88000340bb80 0000000000000046 ffff88002550d000 0000000000000000
ffff88002550d000 0000000000000007 ffff88000340bfd8 ffff88002550d2a8
000000000000ddf0 00000000000118c0 00000000000118c0 ffff88002550d2a8
Call Trace:
[] ? trace_hardirqs_on+0xd/0xf
[] ? cachefiles_wait_bit+0x0/0xd [cachefiles]
[] cachefiles_wait_bit+0x9/0xd [cachefiles]
[] __wait_on_bit+0x43/0x76
[] ? ext3_xattr_get+0x1ec/0x270
[] out_of_line_wait_on_bit+0x69/0x74
[] ? cachefiles_wait_bit+0x0/0xd [cachefiles]
[] ? wake_bit_function+0x0/0x2e
[] cachefiles_mark_object_active+0x203/0x23b [cachefiles]
[] cachefiles_walk_to_object+0x558/0x827 [cachefiles]
[] cachefiles_lookup_object+0xac/0x12a [cachefiles]
[] fscache_lookup_object+0x1c7/0x214 [fscache]
[] fscache_object_state_machine+0xa5/0x52d [fscache]
[] fscache_object_slow_work_execute+0x5f/0xa0 [fscache]
[] slow_work_execute+0x18f/0x2d1
[] slow_work_thread+0x1c5/0x308
[] ? autoremove_wake_function+0x0/0x34
[] ? slow_work_thread+0x0/0x308
[] kthread+0x7a/0x82
[] child_rip+0xa/0x20
[] ? restore_args+0x0/0x30
[] ? kthread+0x0/0x82
[] ? child_rip+0x0/0x20
1 lock held by kslowd004/5711:
#0: (&sb->s_type->i_mutex_key#7/1){+.+.+.}, at: [] cachefiles_walk_to_object+0x1b3/0x827 [cachefiles]Signed-off-by: David Howells
-
Show more debugging information if cachefiles_mark_object_active() is asked to
activate an active object.This may happen, for instance, if the netfs tries to register an object with
the same key multiple times.The code is changed to (a) get the appropriate object lock to protect the
cookie pointer whilst we dereference it, and (b) get and display the cookie key
if available.Signed-off-by: David Howells
-
Mark parent directory locks as I_MUTEX_PARENT in the callers of
cachefiles_bury_object() so that lockdep doesn't complain when that invokes
vfs_unlink():=============================================
[ INFO: possible recursive locking detected ]
2.6.32-rc6-cachefs #47
---------------------------------------------
kslowd002/3089 is trying to acquire lock:
(&sb->s_type->i_mutex_key#7){+.+.+.}, at: [] vfs_unlink+0x8b/0x128but task is already holding lock:
(&sb->s_type->i_mutex_key#7){+.+.+.}, at: [] cachefiles_walk_to_object+0x1b0/0x831 [cachefiles]other info that might help us debug this:
1 lock held by kslowd002/3089:
#0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [] cachefiles_walk_to_object+0x1b0/0x831 [cachefiles]stack backtrace:
Pid: 3089, comm: kslowd002 Not tainted 2.6.32-rc6-cachefs #47
Call Trace:
[] __lock_acquire+0x1649/0x16e3
[] ? inode_has_perm+0x5f/0x61
[] lock_acquire+0x57/0x6d
[] ? vfs_unlink+0x8b/0x128
[] mutex_lock_nested+0x54/0x292
[] ? vfs_unlink+0x8b/0x128
[] ? selinux_inode_permission+0x8e/0x90
[] ? security_inode_permission+0x1c/0x1e
[] ? inode_permission+0x99/0xa5
[] vfs_unlink+0x8b/0x128
[] ? kfree+0xed/0xf9
[] cachefiles_bury_object+0xb6/0x420 [cachefiles]
[] ? trace_hardirqs_on+0xd/0xf
[] ? cachefiles_check_object_xattr+0x233/0x293 [cachefiles]
[] cachefiles_walk_to_object+0x4ff/0x831 [cachefiles]
[] ? finish_task_switch+0x0/0xb2
[] cachefiles_lookup_object+0xac/0x12a [cachefiles]
[] fscache_lookup_object+0x1c7/0x214 [fscache]
[] fscache_object_state_machine+0xa5/0x52d [fscache]
[] fscache_object_slow_work_execute+0x5f/0xa0 [fscache]
[] slow_work_execute+0x18f/0x2d1
[] slow_work_thread+0x1c5/0x308
[] ? autoremove_wake_function+0x0/0x34
[] ? slow_work_thread+0x0/0x308
[] kthread+0x7a/0x82
[] child_rip+0xa/0x20
[] ? restore_args+0x0/0x30
[] ? kthread+0x0/0x82
[] ? child_rip+0x0/0x20Signed-off-by: Daivd Howells
-
Handle truncate unlocking the page we're attempting to read from the backing
device before the read has completed.This was causing reports like the following to occur:
Pid: 4765, comm: kslowd Not tainted 2.6.30.1 #1
Call Trace:
[] ? cachefiles_read_waiter+0xd9/0x147 [cachefiles]
[] ? __wait_on_bit+0x60/0x6f
[] ? __wake_up_common+0x3f/0x71
[] ? __wake_up+0x30/0x44
[] ? __wake_up_bit+0x28/0x2d
[] ? ext3_truncate+0x4d7/0x8ed [ext3]
[] ? pagevec_lookup+0x17/0x1f
[] ? unmap_mapping_range+0x59/0x1ff
[] ? __wake_up+0x30/0x44
[] ? vmtruncate+0xc2/0xe2
[] ? inode_setattr+0x22/0x10a
[] ? ext3_setattr+0x17b/0x1e6 [ext3]
[] ? notify_change+0x186/0x2c9
[] ? cachefiles_attr_changed+0x133/0x1cd [cachefiles]
[] ? cachefiles_lookup_object+0xcf/0x12a [cachefiles]
[] ? fscache_lookup_object+0x110/0x122 [fscache]
[] ? fscache_object_slow_work_execute+0x590/0x6bc
[fscache]
[] ? slow_work_thread+0x285/0x43a
[] ? autoremove_wake_function+0x0/0x2e
[] ? slow_work_thread+0x0/0x43a
[] ? kthread+0x54/0x81
[] ? child_rip+0xa/0x20
[] ? kthread+0x0/0x81
[] ? child_rip+0x0/0x20
CacheFiles: I/O Error: Readpage failed on backing file 200000000000810
FS-Cache: Cache cachefiles stopped due to I/O errorReported-by: Christian Kujau
Reported-by: Takashi Iwai
Reported-by: Duc Le Minh
Signed-off-by: David Howells -
cachefiles_write_page() writes a full page to the backing file for the last
page of the netfs file, even if the netfs file's last page is only a partial
page.This causes the EOF on the backing file to be extended beyond the EOF of the
netfs, and thus the backing file will be truncated by cachefiles_attr_changed()
called from cachefiles_lookup_object().So we need to limit the write we make to the backing file on that last page
such that it doesn't push the EOF too far.Also, if a backing file that has a partial page at the end is expanded, we
discard the partial page and refetch it on the basis that we then have a hole
in the file with invalid data, and should the power go out... A better way to
deal with this could be to record a note that the partial page contains invalid
data until the correct data is written into it.This isn't a problem for netfs's that discard the whole backing file if the
file size changes (such as NFS).Signed-off-by: David Howells
-
Allow the current state of all fscache objects to be dumped by doing:
cat /proc/fs/fscache/objects
By default, all objects and all fields will be shown. This can be restricted
by adding a suitable key to one of the caller's keyrings (such as the session
keyring):keyctl add user fscache:objlist "" @s
The are:
K Show hexdump of object key (don't show if not given)
A Show hexdump of object aux data (don't show if not given)And paired restrictions:
C Show objects that have a cookie
c Show objects that don't have a cookie
B Show objects that are busy
b Show objects that aren't busy
W Show objects that have pending writes
w Show objects that don't have pending writes
R Show objects that have outstanding reads
r Show objects that don't have outstanding reads
S Show objects that have slow work queued
s Show objects that don't have slow work queuedIf neither side of a restriction pair is given, then both are implied. For
example:keyctl add user fscache:objlist KB @s
shows objects that are busy, and lists their object keys, but does not dump
their auxiliary data. It also implies "CcWwRrSs", but as 'B' is given, 'b' is
not implied.Signed-off-by: David Howells