09 May, 2013
1 commit
-
Pull f2fs updates from Jaegeuk Kim:
"This patch-set includes the following major enhancement patches.
- introduce a new gloabl lock scheme
- add tracepoints on several major functions
- fix the overall cleaning process focused on victim selection
- apply the block plugging to merge IOs as much as possible
- enhance management of free nids and its list
- enhance the readahead mode for node pages
- address several cretical deadlock conditions
- reduce lock_page callsThe other minor bug fixes and enhancements are as follows.
- calculation mistakes: overflow
- bio types: READ, READA, and READ_SYNC
- fix the recovery flow, data races, and null pointer errors"* tag 'f2fs-for-v3.10' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs: (68 commits)
f2fs: cover free_nid management with spin_lock
f2fs: optimize scan_nat_page()
f2fs: code cleanup for scan_nat_page() and build_free_nids()
f2fs: bugfix for alloc_nid_failed()
f2fs: recover when journal contains deleted files
f2fs: continue to mount after failing recovery
f2fs: avoid deadlock during evict after f2fs_gc
f2fs: modify the number of issued pages to merge IOs
f2fs: remove useless #include as we're now using sysfs as debug entry.
f2fs: fix inconsistent using of NM_WOUT_THRESHOLD
f2fs: check truncation of mapping after lock_page
f2fs: enhance alloc_nid and build_free_nids flows
f2fs: add a tracepoint on f2fs_new_inode
f2fs: check nid == 0 in add_free_nid
f2fs: add REQ_META about metadata requests for submit
f2fs: give a chance to merge IOs by IO scheduler
f2fs: avoid frequent background GC
f2fs: add tracepoints to debug checkpoint request
f2fs: add tracepoints for write page operations
f2fs: add tracepoints to debug the block allocation
...
08 May, 2013
8 commits
-
After build_free_nids() searches free nid candidates from nat pages and
current journal blocks, it checks all the candidates if they are allocated
so that the nat cache has its nid with an allocated block address.In this procedure, previously we used
list_for_each_entry_safe(fnid, next_fnid, &nm_i->free_nid_list, list).
But, this is not covered by free_nid_list_lock, resulting in null pointer bug.This patch moves this checking routine inside add_free_nid() in order not to use
the spin_lock.Signed-off-by: Jaegeuk Kim
-
When nm_i->fcnt > 2 * MAX_FREE_NIDS, stop scanning other NAT entries.
Signed-off-by: Haicheng Li
[Jaegeuk Kim: fix handling the return value of add_free_nid()]
Signed-off-by: Jaegeuk Kim -
This patch does two cleanups:
1. remove unused variable "fcnt" in build_free_nids().
2. make scan_nat_page() as void type and remove useless variable "fcnt".Signed-off-by: Haicheng Li
Signed-off-by: Jaegeuk Kim -
Directly drop the free_nid cache when nm_i->fcnt > 2 * MAX_FREE_NIDS
Since there is NOT nmi->free_nid_list_lock spinlock protection between
a sequential calling of alloc_nid() and alloc_nid_failed(), some other
threads may already add new free_nid to the free_nid_list during this
period.We need to make sure nmi->fcnt is never > 2 * MAX_FREE_NIDS.
Signed-off-by: Haicheng Li
[Jaegeuk Kim: fit the coding style]
Signed-off-by: Jaegeuk Kim -
When recovering a journal file with fsync data for files that have
been deleted, don't bail out on recovery.Signed-off-by: Chris Fries
Reviewed-by: Russell Knize
Reviewed-by: Jason Hrycay
[Jaegeuk Kim: fit the coding style]
Signed-off-by: Jaegeuk Kim -
When unable to roll forward the journal, we shouldn't bail out and
not mount, we should continue to attempt the mount. Bad recovery data
is likely unrecoverable at this point, and requiring the user to try
to mount again doesn't solve any issues.Signed-off-by: Chris Fries
Reviewed-by: Russell Knize
Reviewed-by: Jason Hrycay
Signed-off-by: Jaegeuk Kim -
o Deadlock case #1
Thread 1:
- writeback_sb_inodes
- do_writepages
- f2fs_write_data_pages
- write_cache_pages
- f2fs_write_data_page
- f2fs_balance_fs
- wait mutex_lock(gc_mutex)Thread 2:
- f2fs_balance_fs
- mutex_lock(gc_mutex)
- f2fs_gc
- f2fs_iget
- wait iget_locked(inode->i_lock)Thread 3:
- do_unlinkat
- iput
- lock(inode->i_lock)
- evict
- inode_wait_for_writebacko Deadlock case #2
Thread 1:
- __writeback_single_inode
: set I_SYNC
- do_writepages
- f2fs_write_data_page
- f2fs_balance_fs
- f2fs_gc
- iput
- evict
- inode_wait_for_writeback(I_SYNC)In order to avoid this, even though iput is called with the zero-reference
count, we need to stop the eviction procedure if the inode is on writeback.
So this patch links f2fs_drop_inode which checks the I_SYNC flag.Signed-off-by: Jaegeuk Kim
-
Faster kernel compiles by way of fewer unnecessary includes.
[akpm@linux-foundation.org: fix fallout]
[akpm@linux-foundation.org: fix build]
Signed-off-by: Kent Overstreet
Cc: Zach Brown
Cc: Felipe Balbi
Cc: Greg Kroah-Hartman
Cc: Mark Fasheh
Cc: Joel Becker
Cc: Rusty Russell
Cc: Jens Axboe
Cc: Asai Thambi S P
Cc: Selvan Mani
Cc: Sam Bradshaw
Cc: Jeff Moyer
Cc: Al Viro
Cc: Benjamin LaHaise
Reviewed-by: "Theodore Ts'o"
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
02 May, 2013
1 commit
-
Pull VFS updates from Al Viro,
Misc cleanups all over the place, mainly wrt /proc interfaces (switch
create_proc_entry to proc_create(), get rid of the deprecated
create_proc_read_entry() in favor of using proc_create_data() and
seq_file etc).7kloc removed.
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (204 commits)
don't bother with deferred freeing of fdtables
proc: Move non-public stuff from linux/proc_fs.h to fs/proc/internal.h
proc: Make the PROC_I() and PDE() macros internal to procfs
proc: Supply a function to remove a proc entry by PDE
take cgroup_open() and cpuset_open() to fs/proc/base.c
ppc: Clean up scanlog
ppc: Clean up rtas_flash driver somewhat
hostap: proc: Use remove_proc_subtree()
drm: proc: Use remove_proc_subtree()
drm: proc: Use minor->index to label things, not PDE->name
drm: Constify drm_proc_list[]
zoran: Don't print proc_dir_entry data in debug
reiserfs: Don't access the proc_dir_entry in r_open(), r_start() r_show()
proc: Supply an accessor for getting the data from a PDE's parent
airo: Use remove_proc_subtree()
rtl8192u: Don't need to save device proc dir PDE
rtl8187se: Use a dir under /proc/net/r8180/
proc: Add proc_mkdir_data()
proc: Move some bits from linux/proc_fs.h to linux/{of.h,signal.h,tty.h}
proc: Move PDE_NET() to fs/proc/proc_net.c
...
01 May, 2013
1 commit
-
Pull trivial tree updates from Jiri Kosina:
"Usual stuff, mostly comment fixes, typo fixes, printk fixes and small
code cleanups"* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (45 commits)
mm: Convert print_symbol to %pSR
gfs2: Convert print_symbol to %pSR
m32r: Convert print_symbol to %pSR
iostats.txt: add easy-to-find description for field 6
x86 cmpxchg.h: fix wrong comment
treewide: Fix typo in printk and comments
doc: devicetree: Fix various typos
docbook: fix 8250 naming in device-drivers
pata_pdc2027x: Fix compiler warning
treewide: Fix typo in printks
mei: Fix comments in drivers/misc/mei
treewide: Fix typos in kernel messages
pm44xx: Fix comment for "CONFIG_CPU_IDLE"
doc: Fix typo "CONFIG_CGROUP_CGROUP_MEMCG_SWAP"
mmzone: correct "pags" to "pages" in comment.
kernel-parameters: remove outdated 'noresidual' parameter
Remove spurious _H suffixes from ifdef comments
sound: Remove stray pluses from Kconfig file
radio-shark: Fix printk "CONFIG_LED_CLASS"
doc: put proper reference to CONFIG_MODULE_SIG_ENFORCE
...
30 Apr, 2013
3 commits
-
When testing f2fs on an SSD, I found some 128 page IOs followed by 1 page IO
were issued by f2fs_write_node_pages.
This means that there were some mishandling flows which degrades performance.Previous f2fs_write_node_pages determines the number of pages to be written,
nr_to_write, as follows.1. The bio_get_nr_vecs returns 129 pages.
2. The bio_alloc makes a room for 128 pages.
3. The initial 128 pages go into one bio.
4. The existing bio is submitted, and a new bio is prepared for the last 1 page.
5. Finally, sync_node_pages submits the last 1 page bio.The problem is from the use of bio_get_nr_vecs, so this patch replace it
with max_hw_blocks using queue_max_sectors.Signed-off-by: Jaegeuk Kim
-
Signed-off-by: Haicheng Li
Signed-off-by: Jaegeuk Kim -
try_to_free_nats() is usually called with parameter nr_shrink as
"nm_i->nat_cnt - NM_WOUT_THRESHOLD"
by flush_nat_entries() during checkpointing process.However, this is inconsistent with the actual threshold check as
"if (nm_i->nat_cnt < 2 * NM_WOUT_THRESHOLD)"
, which will ignore the free_nats requests when
NM_WOUT_THRESHOLD < nm_i->nat_cnt < 2 * NM_WOUT_THRESHOLDSo fix the threshold check condition.
Signed-off-by: Haicheng Li
Signed-off-by: Jaegeuk Kim
29 Apr, 2013
3 commits
-
We call lock_page when we need to update a page after readpage.
Between grab and lock page, the page can be truncated by other thread.
So, we should check the page after lock_page whether it was truncated or not.Signed-off-by: Jaegeuk Kim
-
In order to avoid build_free_nid lock contention, let's change the order of
function calls as follows.At first, check whether there is enough free nids.
- If available, just get a free nid with spin_lock without any overhead.
- Otherwise, conduct build_free_nids.
: scan nat pages, journal nat entries, and nat cache entries.We should consider carefullly not to serve free nids intermediately made by
build_free_nids.
We can get stable free nids only after build_free_nids is done.Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim -
This can help when debugging the free nid allocation flows.
Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim
26 Apr, 2013
4 commits
-
It is more obvious that add_free_nid checks whether the free nid is zero or not.
Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim -
Adding REQ_META for all the metadata requests can help in improving the
FS performance, if the underlying device supports TAGGING.
So, when considering the submit_bio path for all the f2fs requests. We can
add REQ_META for all the META requests.
As a precursor to this change we considered the commit
4265900e0be653f5b78baf2816857ef57cf1332f 'mmc: MMC-4.5 Data Tag Support'Signed-off-by: Namjae Jeon
Signed-off-by: Amit Sahrawat
Signed-off-by: Jaegeuk Kim -
Previously, background GC submits many 4KB read requests to load victim blocks
and/or its (i)node blocks....
f2fs_gc : f2fs_readpage: ino = 1, page_index = 0xb61, blkaddr = 0x3b964ed
f2fs_gc : block_rq_complete: 8,16 R () 499854968 + 8 [0]
f2fs_gc : f2fs_readpage: ino = 1, page_index = 0xb6f, blkaddr = 0x3b964ee
f2fs_gc : block_rq_complete: 8,16 R () 499854976 + 8 [0]
f2fs_gc : f2fs_readpage: ino = 1, page_index = 0xb79, blkaddr = 0x3b964ef
f2fs_gc : block_rq_complete: 8,16 R () 499854984 + 8 [0]
...However, by the fact that many IOs are sequential, we can give a chance to merge
the IOs by IO scheduler.
In order to do that, let's use blk_plug....
f2fs_gc : f2fs_iget: ino = 143
f2fs_gc : f2fs_readpage: ino = 143, page_index = 0x1c6, blkaddr = 0x2e6ee
f2fs_gc : f2fs_iget: ino = 143
f2fs_gc : f2fs_readpage: ino = 143, page_index = 0x1c7, blkaddr = 0x2e6ef
: block_rq_complete: 8,16 R () 1519616 + 8 [0]
: block_rq_complete: 8,16 R () 1519848 + 8 [0]
: block_rq_complete: 8,16 R () 1520432 + 96 [0]
: block_rq_complete: 8,16 R () 1520536 + 104 [0]
: block_rq_complete: 8,16 R () 1521008 + 112 [0]
: block_rq_complete: 8,16 R () 1521440 + 152 [0]
: block_rq_complete: 8,16 R () 1521688 + 144 [0]
: block_rq_complete: 8,16 R () 1522128 + 192 [0]
: block_rq_complete: 8,16 R () 1523256 + 328 [0]
...Note that this issue should be addressed in checkpoint, and some readahead
flows too.Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim -
If there is no victim segments selected by background GC, let's wait
a little bit longer time to collect dirty segments.
By default, let's give 5 minutes.Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim
23 Apr, 2013
8 commits
-
Add tracepoints to debug checkpoint request.
Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: change expressions]
Signed-off-by: Jaegeuk Kim -
Add tracepoints to debug the various page write operation
like data pages, meta pages.Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: remove unnecessary tracepoints]
Signed-off-by: Jaegeuk Kim -
Add tracepoints to debug the block allocation & fallocate.
Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: enhance information]
Signed-off-by: Jaegeuk Kim -
Add tracepoints for tracing the garbage collector
threads in f2fs with status of collection & type.Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: modify slightly to show information]
Signed-off-by: Jaegeuk Kim -
Add tracepoints for page i/o operations and block allocation
tracing during page read operation.Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: combine and modify the tracepoint structures]
Signed-off-by: Jaegeuk Kim -
add tracepoints for tracing the truncate operations
like truncate node/data blocks, f2fs_truncate etc.Tracepoints are added at entry and exit of operation
to trace the success & failure of operation.Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: combine and modify the tracepoint structures]
Signed-off-by: Jaegeuk Kim -
Add tracepoints in f2fs for tracing the syncing
operations like filesystem sync, file sync enter/exit.
It will helf to trace the code under debugging scenarios.Also add tracepoints for tracing the various inode operations
like building inode, eviction of inode, link/unlike of
inodes.Signed-off-by: Namjae Jeon
Signed-off-by: Pankaj Kumar
Acked-by: Steven Rostedt
[Jaegeuk: combine and modify the tracepoint structures]
Signed-off-by: Jaegeuk Kim -
The code conditions put inside the function is_multimedia_file are
reverse to the name i.e, we need to negate the return to actually
check if the file is a multimedia file. So, change the code and usage
path to align both the name and comparision conditions.Signed-off-by: Namjae Jeon
Signed-off-by: Amit Sahrawat
Signed-off-by: Jaegeuk Kim
22 Apr, 2013
1 commit
-
Fix to return a negative error code from the error handling
case instead of 0, as returned elsewhere in this function.
Introduce by commit c0d39e(f2fs: fix return values from validate superblock)Signed-off-by: Wei Yongjun
Acked-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim
10 Apr, 2013
2 commits
-
Signed-off-by: Al Viro
-
Signed-off-by: Al Viro
09 Apr, 2013
5 commits
-
Fix typo mistakes.
1. I think that it should be 'L' instead of 'V'.
2. and try to fix 'Front' instead of 'Frone'Signed-off-by: Namjae Jeon
Signed-off-by: Amit Sahrawat
Signed-off-by: Jaegeuk Kim -
In order to be aware of prefree and free sections during FG_GC, let's start with
write_checkpoint().Signed-off-by: Jaegeuk Kim
-
If (ofs % (NIDS_PER_BLOCK + 1) == 0), the node is an indirect node block.
Signed-off-by: Zhihui Zhang
Signed-off-by: Jaegeuk Kim -
In the previous version, f2fs uses global locks according to the usage types,
such as directory operations, block allocation, block write, and so on.Reference the following lock types in f2fs.h.
enum lock_type {
RENAME, /* for renaming operations */
DENTRY_OPS, /* for directory operations */
DATA_WRITE, /* for data write */
DATA_NEW, /* for data allocation */
DATA_TRUNC, /* for data truncate */
NODE_NEW, /* for node allocation */
NODE_TRUNC, /* for node truncate */
NODE_WRITE, /* for node write */
NR_LOCK_TYPE,
};In that case, we lose the performance under the multi-threading environment,
since every types of operations must be conducted one at a time.In order to address the problem, let's share the locks globally with a mutex
array regardless of any types.
So, let users grab a mutex and perform their jobs in parallel as much as
possbile.For this, I propose a new global lock scheme as follows.
0. Data structure
- f2fs_sb_info -> mutex_lock[NR_GLOBAL_LOCKS]
- f2fs_sb_info -> node_write1. mutex_lock_op(sbi)
- try to get an avaiable lock from the array.
- returns the index of the gottern lock variable.2. mutex_unlock_op(sbi, index of the lock)
- unlock the given index of the lock.3. mutex_lock_all(sbi)
- grab all the locks in the array before the checkpoint.4. mutex_unlock_all(sbi)
- release all the locks in the array after checkpoint.5. block_operations()
- call mutex_lock_all()
- sync_dirty_dir_inodes()
- grab node_write
- sync_node_pages()Note that,
the pairs of mutex_lock_op()/mutex_unlock_op() and
mutex_lock_all()/mutex_unlock_all() should be used together.Signed-off-by: Jaegeuk Kim
-
Move the f2fs_balance_fs out of the truncate_hole function and only
perform that in punch_hole use case. The commit:ed60b1644e7f7e5dd67d21caf7e4425dff05dad0
intended to do this but moved it into truncate_hole to cover more
cases. However, a deadlock scenario is possible when deleting an inode
entry under specific conditions:f2fs_delete_entry()
mutex_lock_op(sbi, DENTRY_OPS);
truncate_hole()
f2fs_balance_fs()
mutex_lock(&sbi->gc_mutex);
f2fs_gc()
write_checkpoint()
block_operations()
mutex_lock_op(sbi, DENTRY_OPS);Lets move it into the punch_hole case to cover the original intent of
avoiding it during fallocate's expand_inode_data case.Change-Id: I29f8ea1056b0b88b70ba8652d901b6e8431bb27e
Signed-off-by: Jason Hrycay
Signed-off-by: Jaegeuk Kim
03 Apr, 2013
3 commits
-
This patch reduces redundant spin_lock operations in alloc_nid_failed().
The alloc_nid_failed() does not need to delete entry and add one again
by triggering spin_lock and spin_unlock redundantly.Signed-off-by: Jaegeuk Kim
-
Commit - fa9150a84c - replaces a call to generic_writepages() in
f2fs_write_data_pages() with write_cache_pages(), with a function pointer
argument pointing to routine: __f2fs_writepage.-> https://git.kernel.org/linus/fa9150a84ca333f68127097c4fa1eda4b3913a22
This patch adds a NULL pointer check in f2fs_write_data_pages() to avoid
a possible NULL pointer dereference, in case if - mapping->a_ops->writepage -
is NULL.Signed-off-by: P J P
Signed-off-by: Jaegeuk Kim -
Like below, there are 8 segment bitmaps for SSR victim candidates.
enum dirty_type {
DIRTY_HOT_DATA, /* dirty segments assigned as hot data logs */
DIRTY_WARM_DATA, /* dirty segments assigned as warm data logs */
DIRTY_COLD_DATA, /* dirty segments assigned as cold data logs */
DIRTY_HOT_NODE, /* dirty segments assigned as hot node logs */
DIRTY_WARM_NODE, /* dirty segments assigned as warm node logs */
DIRTY_COLD_NODE, /* dirty segments assigned as cold node logs */
DIRTY, /* to count # of dirty segments */
PRE, /* to count # of entirely obsolete segments */
NR_DIRTY_TYPE
};The upper 6 bitmaps indicates segments dirtied by active log areas respectively.
And, the DIRTY bitmap integrates all the 6 bitmaps.For example,
o DIRTY_HOT_DATA : 1010000
o DIRTY_WARM_DATA: 0100000
o DIRTY_COLD_DATA: 0001000
o DIRTY_HOT_NODE : 0000010
o DIRTY_WARM_NODE: 0000001
o DIRTY_COLD_NODE: 0000000
In this case,
o DIRTY : 1111011,which means that we should guarantee the consistency between DIRTY and other
bitmaps concreately.However, the SSR mode selects victims freely from any log types, which can set
multiple bits across the various bitmap types.So, this patch eliminates this inconsistency.
Reviewed-by: Namjae Jeon
Signed-off-by: Jaegeuk Kim
