29 Mar, 2012
1 commit
-
Pull trivial exofs changes from Boaz Harrosh:
"Just nothingness really. The big exofs changes are reserved for the
next merge window."* 'for-linus' of git://git.open-osd.org/linux-open-osd:
exofs: Cap on the memcpy() size
exofs: (trivial) Fix typo in super.c
exofs: fix endian conversion in exofs_sync_fs()
22 Mar, 2012
1 commit
-
Pull vfs pile 1 from Al Viro:
"This is _not_ all; in particular, Miklos' and Jan's stuff is not there
yet."* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (64 commits)
ext4: initialization of ext4_li_mtx needs to be done earlier
debugfs-related mode_t whack-a-mole
hfsplus: add an ioctl to bless files
hfsplus: change finder_info to u32
hfsplus: initialise userflags
qnx4: new helper - try_extent()
qnx4: get rid of qnx4_bread/qnx4_getblk
take removal of PF_FORKNOEXEC to flush_old_exec()
trim includes in inode.c
um: uml_dup_mmap() relies on ->mmap_sem being held, but activate_mm() doesn't hold it
um: embed ->stub_pages[] into mmu_context
gadgetfs: list_for_each_safe() misuse
ocfs2: fix leaks on failure exits in module_init
ecryptfs: make register_filesystem() the last potential failure exit
ntfs: forgets to unregister sysctls on register_filesystem() failure
logfs: missing cleanup on register_filesystem() failure
jfs: mising cleanup on register_filesystem() failure
make configfs_pin_fs() return root dentry on success
configfs: configfs_create_dir() has parent dentry in dentry->d_parent
configfs: sanitize configfs_create()
...
21 Mar, 2012
2 commits
-
Signed-off-by: Al Viro
-
New field of struct super_block - ->s_max_links. Maximal allowed
value of ->i_nlink or 0; in the latter case all checks still need
to be done in ->link/->mkdir/->rename instances. Note that this
limit applies both to directoris and to non-directories.Signed-off-by: Al Viro
20 Mar, 2012
4 commits
-
Ack-by: Boaz Harrosh
Signed-off-by: Cong Wang -
This data comes from the device, so probably it's fairly trustworthy but
it makes the static checkers happy if we check it.[Boaz]
the system_id_len is zero, if not present, or always OSD_SYSTEMID_LEN.
So always copy OSD_SYSTEMID_LEN bytes.Signed-off-by: Dan Carpenter
Signed-off-by: Boaz Harrosh -
Correct spelling "faild" to "failed" in
fs/exofs/super.cSigned-off-by: Masanari Iida
Signed-off-by: Boaz Harrosh -
fscb->s_numfiles is an __le64 field so we need to use cpu_to_le64()
to get a little endian 64 bit on big endian systems.Signed-off-by: Dan Carpenter
Signed-off-by: Boaz Harrosh
10 Jan, 2012
1 commit
-
* 'for-linus' of git://git.open-osd.org/linux-open-osd:
ore: Must support none-PAGE-aligned IO
ore: fix BUG_ON, too few sgs when reading
ore: Fix crash in case of an IO error.
ore: FIX breakage when MISC_FILESYSTEMS is not set
09 Jan, 2012
1 commit
-
We have already set ->s_root, so ->put_super() is going to be called.
Freeing ->s_fs_info is a bloody bad idea when it's going to be
dereferenced very shortly...Signed-off-by: Al Viro
08 Jan, 2012
1 commit
-
NFS might send us offsets that are not PAGE aligned. So
we must read in the reminder of the first/last pages, in cases
we need it for Parity calculations.We only add an sg segments to read the partial page. But
we don't mark it as read=true because it is a lock-for-write
page.TODO: In some cases (IO spans a single unit) we can just
adjust the raid_unit offset/length, but this is left for
later Kernels.[Bug in 3.2.0 Kernel]
CC: Stable Tree
Signed-off-by: Boaz Harrosh
06 Jan, 2012
3 commits
-
When reading RAID5 files, in rare cases, we calculated too
few sg segments. There should be two extra for the beginning
and end partial units.Also "too few sg segments" should not be a BUG_ON there is
all the mechanics in place to handle it, as a short read.
So just return -ENOMEM and the rest of the code will gracefully
split the IO.[Bug in 3.2.0 Kernel]
CC: Stable Tree
Signed-off-by: Boaz Harrosh -
The users of ore_check_io() expect the reported device
(In case of error) to be indexed relative to the passed-in
ore_components table, and not the logical dev index.This causes a crash inside objlayoutdriver in case of
an IO error.[Bug in 3.2.0 Kernel]
CC: Stable Tree
Signed-off-by: Boaz Harrosh -
As Reported by Randy Dunlap
When MISC_FILESYSTEMS is not enabled and NFS4.1 is:
fs/built-in.o: In function `objio_alloc_io_state':
objio_osd.c:(.text+0xcb525): undefined reference to `ore_get_rw_state'
fs/built-in.o: In function `_write_done':
objio_osd.c:(.text+0xcb58d): undefined reference to `ore_check_io'
fs/built-in.o: In function `_read_done':
...When MISC_FILESYSTEMS, which is more of a GUI thing then anything else,
is not selected. exofs/Kconfig is never examined during Kconfig,
and it can not do it's magic stuff to automatically select everything
needed.We must split exofs/Kconfig in two. The ore one is always included.
And the exofs one is left in it's old place in the menu.[Needed for the 3.2.0 Kernel]
CC: Stable Tree
Reported-by: Randy Dunlap
Signed-off-by: Boaz Harrosh
04 Jan, 2012
5 commits
-
Signed-off-by: Al Viro
-
Signed-off-by: Al Viro
-
vfs_create() ignores everything outside of 16bit subset of its
mode argument; switching it to umode_t is obviously equivalent
and it's the only caller of the methodSigned-off-by: Al Viro
-
vfs_mkdir() gets int, but immediately drops everything that might not
fit into umode_t and that's the only caller of ->mkdir()...Signed-off-by: Al Viro
-
Seeing that just about every destructor got that INIT_LIST_HEAD() copied into
it, there is no point whatsoever keeping this INIT_LIST_HEAD in inode_init_once();
the cost of taking it into inode_init_always() will be negligible for pipes
and sockets and negative for everything else. Not to mention the removal of
boilerplate code from ->destroy_inode() instances...Signed-off-by: Al Viro
07 Nov, 2011
1 commit
-
* 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux: (230 commits)
Revert "tracing: Include module.h in define_trace.h"
irq: don't put module.h into irq.h for tracking irqgen modules.
bluetooth: macroize two small inlines to avoid module.h
ip_vs.h: fix implicit use of module_get/module_put from module.h
nf_conntrack.h: fix up fallout from implicit moduleparam.h presence
include: replace linux/module.h with "struct module" wherever possible
include: convert various register fcns to macros to avoid include chaining
crypto.h: remove unused crypto_tfm_alg_modname() inline
uwb.h: fix implicit use of asm/page.h for PAGE_SIZE
pm_runtime.h: explicitly requires notifier.h
linux/dmaengine.h: fix implicit use of bitmap.h and asm/page.h
miscdevice.h: fix up implicit use of lists and types
stop_machine.h: fix implicit use of smp.h for smp_processor_id
of: fix implicit use of errno.h in include/linux/of.h
of_platform.h: delete needless include
acpi: remove module.h include from platform/aclinux.h
miscdevice.h: delete unnecessary inclusion of module.h
device_cgroup.h: delete needless include
net: sch_generic remove redundant use of
net: inet_timewait_sock doesnt need
...Fix up trivial conflicts (other header files, and removal of the ab3550 mfd driver) in
- drivers/media/dvb/frontends/dibx000_common.c
- drivers/media/video/{mt9m111.c,ov6650.c}
- drivers/mfd/ab3550-core.c
- include/linux/dmaengine.h
05 Nov, 2011
1 commit
-
* 'nfs-for-3.2' of git://git.linux-nfs.org/projects/trondmy/linux-nfs: (25 commits)
nfs: set vs_hidden on nfs4_callback_version4 (try #2)
pnfs-obj: Support for RAID5 read-4-write interface.
pnfs-obj: move to ore 03: Remove old raid engine
pnfs-obj: move to ore 02: move to ORE
pnfs-obj: move to ore 01: ore_layout & ore_components
pnfs-obj: Rename objlayout_io_state => objlayout_io_res
pnfs-obj: Get rid of objlayout_{alloc,free}_io_state
pnfs-obj: Return PNFS_NOT_ATTEMPTED in case of read/write_pagelist
pnfs-obj: Remove redundant EOF from objlayout_io_state
nfs: Remove unused variable from write.c
nfs: Fix unused variable warning from file.c
NFS: Remove no-op less-than-zero checks on unsigned variables.
NFS: Clean up nfs4_xdr_dec_secinfo()
NFS: Fix documenting comment for nfs_create_request()
NFS4: fix cb_recallany decode error
nfs4: serialize layoutcommit
SUNRPC: remove rpcbind clients destruction on module cleanup
SUNRPC: remove rpcbind clients creation during service registering
NFSd: call svc rpcbind cleanup explicitly
SUNRPC: cleanup service destruction
...
03 Nov, 2011
1 commit
-
In this patch we are actually moving to the ORE.
(Object Raid Engine).objio_state holds a pointer to an ore_io_state. Once
we have an ore_io_state at hand we can call the ore
for reading/writing. We register on the done path
to kick off the nfs io_done mechanism.Again for Ease of reviewing the old code is "#if 0"
but is not removed so the diff command works better.
The old code will be removed in the next patch.fs/exofs/Kconfig::ORE is modified to also be auto-included
if PNFS_OBJLAYOUT is set. Since we now depend on ORE.
(See comments in fs/exofs/Kconfig)Signed-off-by: Boaz Harrosh
Signed-off-by: Trond Myklebust
02 Nov, 2011
1 commit
-
Replace remaining direct i_nlink updates with a new set_nlink()
updater function.Signed-off-by: Miklos Szeredi
Tested-by: Toshiyuki Okajima
Signed-off-by: Christoph Hellwig
01 Nov, 2011
1 commit
-
Some files were using the complete module.h infrastructure without
actually including the header at all. Fix them up in advance so
once the implicit presence is removed, we won't get failures like this:CC [M] fs/nfsd/nfssvc.o
fs/nfsd/nfssvc.c: In function 'nfsd_create_serv':
fs/nfsd/nfssvc.c:335: error: 'THIS_MODULE' undeclared (first use in this function)
fs/nfsd/nfssvc.c:335: error: (Each undeclared identifier is reported only once
fs/nfsd/nfssvc.c:335: error: for each function it appears in.)
fs/nfsd/nfssvc.c: In function 'nfsd':
fs/nfsd/nfssvc.c:555: error: implicit declaration of function 'module_put_and_exit'
make[3]: *** [fs/nfsd/nfssvc.o] Error 1Signed-off-by: Paul Gortmaker
25 Oct, 2011
5 commits
-
Now that we support raid5 Enable it at mount. Raid6 will come next
raid4 is not demanded for so it will probably not be enabled.
(Until some one wants it)NOTE: That mkfs.exofs had support for raid5/6 since long time
ago. (Making an empty raidX FS is just as easy as raid0 ;-} )Signed-off-by: Boaz Harrosh
-
The ore need suplied a r4w_get_page/r4w_put_page API
from Filesystem so it can get cache pages to read-into when
writing parial stripes.Also I commented out and NULLed the .writepage (singular)
vector. Because it gives terrible write pattern to raid
and is apparently not needed. Even in OOM conditions the
system copes (even better) with out it.TODO: How to specify to write_cache_pages() to start
or include a certain page?Signed-off-by: Boaz Harrosh
-
This is finally the RAID5 Write support.
The bigger part of this patch is not the XOR engine itself, But the
read4write logic, which is a complete mini prepare_for_striping
reading engine that can read scattered pages of a stripe into cache
so it can be used for XOR calculation. That is, if the write was not
stripe aligned.The main algorithm behind the XOR engine is the 2 dimensional array:
struct __stripe_pages_2d.
A drawing might save 1000 words
---__stripe_pages_2d
|
n = pages_in_stripe_unit;
w = group_width - parity;
| pages array presented to the XOR lib
| |
V |
__1_page_stripe[0].pages --> [c0][c1]..[cw][c_par] [c0][c1]..[cw][c_par] [c0][c1]..[cw][c_par]
^
|
data added columns first then row---
The pages are put on this array columns first. .i.e:
p0-of-c0, p1-of-c0, ... pn-of-c0, p0-of-c1, ...
So we are doing a corner turn of the pages.Note that pages will zigzag down and left. but are put sequentially
in growing order. So when the time comes to XOR the stripe, only the
beginning and end of the array need be checked. We scan the array
and any NULL spot will be field by pages-to-be-read.The FS that wants to support RAID5 needs to supply an
operations-vector that searches a given page in cache, and specifies
if the page is uptodate or need reading. All these pages to be read
are put on a slave ore_io_state and synchronously read. All the pages
of a stripe are read in one IO, using the scatter gather mechanism.In write we constrain our IO to only be incomplete on a single
stripe. Meaning either the complete IO is within a single stripe so
we might have pages to read from both beginning or end of the
strip. Or we have some reading to do at beginning but end at strip
boundary. The left over pages are pushed to the next IO by the API
already established by previous work, where an IO offset/length
combination presented to the ORE might get the length truncated and
the user must re-submit the leftover pages. (Both exofs and NFS
support this)But any ORE user should make it's best effort to align it's IO
before hand and avoid complications. A cached ore_layout->stripe_size
member can be used for that calculation. (NOTE: that ORE demands
that stripe_size may not be bigger then 32bit)What else? Well read it and tell me.
Signed-off-by: Boaz Harrosh
-
This patch introduces the first stage of RAID5 support
mainly the skip-over-raid-units when reading. For
writes it inserts BLANK units, into where XOR blocks
should be calculated and written to.It introduces the new "general raid maths", and the main
additional parameters and components needed for raid5.Since at this stage it could corrupt future version that
actually do support raid5. The enablement of raid5
mounting and setting of parity-count > 0 is disabled. So
the raid5 code will never be used. Mounting of raid5 is
only enabled later once the basic XOR write is also in.
But if the patch "enable RAID5" is applied this code has
been tested to be able to properly read raid5 volumes
and is according to standard.Also it has been tested that the new maths still properly
supports RAID0 and grouping code just as before.
(BTW: I have found more bugs in the pnfs-obj RAID math
fixed here)The ore.c file is getting too big, so new ore_raid.[hc]
files are added that will include the special raid stuff
that are not used in striping and mirrors. In future write
support these will get bigger.
When adding the ore_raid.c to Kbuild file I was forced to
rename ore.ko to libore.ko. Is it possible to keep source
file, say ore.c and module file ore.ko the same even if there
are multiple files inside ore.ko?Signed-off-by: Boaz Harrosh
-
ore_calc_stripe_info is needed by exofs::export.c
for the layout calculations. Make it exportableSigned-off-by: Boaz Harrosh
15 Oct, 2011
8 commits
-
Current ore_check_io API receives a residual
pointer, to report partial IO. But it is actually
not used, because in a multiple devices IO there
is never a linearity in the IO failure.On the other hand if every failing device is reported
through a received callback measures can be taken to
handle only failed devices. One at a time.This will also be needed by the objects-layout-driver
for it's error reporting facility.Exofs is not currently using the new information and
keeps the old behaviour of failing the complete IO in
case of an error. (No partial completion)TODO: Use an ore_check_io callback to set_page_error only
the failing pages. And re-dirty write pages.Signed-off-by: Boaz Harrosh
-
All users of the ore will need to check if current code
supports the given layout. For example RAID5/6 is not
currently supported.So move all the checks from exofs/super.c to a new
ore_verify_layout() to be used by ore users.Note that any new layout should be passed through the
ore_verify_layout() because the ore engine will prepare
and verify some internal members of ore_layout, and
assumes it's called.Signed-off-by: Boaz Harrosh
-
Users like the objlayout-driver would like to only pass
a partial device table that covers the IO in question.
For example exofs divides the file into raid-group-sized
chunks and only serves group_width number of devices at
a time.The partiality is communicated by setting
ore_componets->first_dev and the array covers all logical
devices from oc->first_dev upto (oc->first_dev + oc->numdevs)The ore_comp_dev() API receives a logical device index
and returns the actual present device in the table.
An out-of-range dev_index will BUG.Logical device index is the theoretical device index as if
all the devices of a file are present. .i.e:
total_devs = group_width * mirror_p1 * group_count
0 < total_devsSigned-off-by: Boaz Harrosh
-
Memory conditions and max_bio constraints might cause us to
not comply to the full length of the requested IO. Instead of
failing the complete IO we can issue a shorter read/write and
report how much was actually executed in the ios->length
member.All users must check ios->length at IO_done or upon return of
ore_read/write and re-issue the reminder of the bytes. Because
other wise there is no error returned like before.This is part of the effort to support the pnfs-obj layout driver.
Signed-off-by: Boaz Harrosh
-
If at read/write_done the actual IO was shorter then requested,
reported in returned ios->length. It is not an error. The reminder
of the pages should just be unlocked but not marked uptodate or
end_page_writeback. They will be re issued later by the VFS.Signed-off-by: Boaz Harrosh
-
Move the check and preparation of the ios->kern_buff case to
later inside _write_mirror().Since read was never used with ios->kern_buff its support is removed
instead of fixed.Signed-off-by: Boaz Harrosh
-
Now that each ore_io_state covers only a single raid group.
A single striping_info math is needed. Embed one inside
ore_io_state to cache the calculation results and eliminate
an extra call.Also the outer _prepare_for_striping is removed since it does nothing.
Signed-off-by: Boaz Harrosh
-
Usually a single IO is confined to one group of devices
(group_width) and at the boundary of a raid group it can
spill into a second group. Current code would allocate a
full device_table size array at each io_state so it can
comply to requests that span two groups. Needless to say
that is very wasteful, specially when device_table count
can get very large (hundreds even thousands), while a
group_width is usually 8 or 10.* Change ore API to trim on IO that spans two raid groups.
The user passes offset+length to ore_get_rw_state, the
ore might trim on that length if spanning a group boundary.
The user must check ios->length or ios->nrpages to see
how much IO will be preformed. It is the responsibility
of the user to re-issue the reminder of the IO.* Modify exofs To copy spilled pages on to the next IO.
This means one last kick is needed after all coalescing
of pages is done.Signed-off-by: Boaz Harrosh
04 Oct, 2011
1 commit
-
In the pNFS obj-LD the device table at the layout level needs
to point to a device_cache node, where it is possible and likely
that many layouts will point to the same device-nodes.In Exofs we have a more orderly structure where we have a single
array of devices that repeats twice for a round-robin view of the
device tableThis patch moves to a model that can be used by the pNFS obj-LD
where struct ore_components holds an array of ore_dev-pointers.
(ore_dev is newly defined and contains a struct osd_dev *od
member)Each pointer in the array of pointers will point to a bigger
user-defined dev_struct. That can be accessed by use of the
container_of macro.In Exofs an __alloc_dev_table() function allocates the
ore_dev-pointers array as well as an exofs_dev array, in one
allocation and does the addresses dance to set everything pointing
correctly. It still keeps the double allocation trick for the
inodes round-robin view of the table.The device table is always allocated dynamically, also for the
single device case. So it is unconditionally freed at umount.Signed-off-by: Boaz Harrosh
03 Oct, 2011
2 commits
-
The struct ore_striping_info will be used later in other
structures. And ore_calc_stripe_info as well. Rename them
make struct ore_striping_info public. ore_calc_stripe_info
is still static, will be made public on first use.Signed-off-by: Boaz Harrosh
-
The struct pnfs_osd_data_map data_map member of exofs_sb_info was
never used after mount. In fact all it's members were duplicated
by the ore_layout structure. So just remove the duplicated information.Also removed some stupid, but perfectly supported, restrictions on
layout parameters. The case where num_devices is not divisible by
mirror_count+1 is perfectly fine since the rotating device view
will eventually use all the devices it can get.Signed-off-by: Boaz Harrosh
Signed-off-by: Benny Halevy