25 Nov, 2015
2 commits
-
Ted and Namjae have reported that truncated pages don't get timely
reclaimed after being truncated in data=journal mode. The following test
triggers the issue easily:for (i = 0; i < 1000; i++) {
pwrite(fd, buf, 1024*1024, 0);
fsync(fd);
fsync(fd);
ftruncate(fd, 0);
}The reason is that journal_unmap_buffer() finds that truncated buffers
are not journalled (jh->b_transaction == NULL), they are part of
checkpoint list of a transaction (jh->b_cp_transaction != NULL) and have
been already written out (!buffer_dirty(bh)). We clean such buffers but
we leave them in the checkpoint list. Since checkpoint transaction holds
a reference to the journal head, these buffers cannot be released until
the checkpoint transaction is cleaned up. And at that point we don't
call release_buffer_page() anymore so pages detached from mapping are
lingering in the system waiting for reclaim to find them and free them.Fix the problem by removing buffers from transaction checkpoint lists
when journal_unmap_buffer() finds out they don't have to be there
anymore.Reported-and-tested-by: Namjae Jeon
Fixes: de1b794130b130e77ffa975bb58cb843744f9ae5
Signed-off-by: Jan Kara
Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org -
In ext4, the bottom two bits of {a,c,m}time_extra are used to extend
the {a,c,m}time fields, deferring the year 2038 problem to the year
2446.When decoding these extended fields, for times whose bottom 32 bits
would represent a negative number, sign extension causes the 64-bit
extended timestamp to be negative as well, which is not what's
intended. This patch corrects that issue, so that the only negative
{a,c,m}times are those between 1901 and 1970 (as per 32-bit signed
timestamps).Some older kernels might have written pre-1970 dates with 1,1 in the
extra bits. This patch treats those incorrectly-encoded dates as
pre-1970, instead of post-2311, until kernel 4.20 is released.
Hopefully by then e2fsck will have fixed up the bad data.Also add a comment explaining the encoding of ext4's extra {a,c,m}time
bits.Signed-off-by: David Turner
Signed-off-by: Theodore Ts'o
Reported-by: Mark Harris
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=23732
Cc: stable@vger.kernel.org
30 Oct, 2015
1 commit
-
As new_valid_dev always returns 1, so !new_valid_dev check is not
needed, remove it.Signed-off-by: Yaowei Bai
Signed-off-by: Theodore Ts'o
19 Oct, 2015
4 commits
-
The ext4_fsblk_t type is a long long, which should not be used
with abs(), as is done in ext4_mb_check_group_pa().This patch modifies ext4_mb_check_group_pa() to use abs64()
instead.Signed-off-by: John Stultz
Signed-off-by: Theodore Ts'o -
It is appeared that we can pass journal related mount options and such options
be shown in /proc/mountsExample:
#mkfs.ext4 -F /dev/vdb
#tune2fs -O ^has_journal /dev/vdb
#mount /dev/vdb /mnt/ -ocommit=20,journal_async_commit
#cat /proc/mounts | grep /mnt
/dev/vdb /mnt ext4 rw,relatime,journal_checksum,journal_async_commit,commit=20,data=ordered 0 0But options:"journal_checksum,journal_async_commit,commit=20,data=ordered" has
nothing with reality because there is no journal at all.This patch disallow following options for journalless configurations:
- journal_checksum
- journal_async_commit
- commit=%ld
- data={writeback,ordered,journal}Signed-off-by: Dmitry Monakhov
Signed-off-by: Theodore Ts'o
Reviewed-by: Andreas Dilger -
Currently MOPT_EXPLICIT treated as EXPLICIT_DELALLOC which may be changed
in future. Let's fix it now.Signed-off-by: Dmitry Monakhov
Signed-off-by: Theodore Ts'o -
If a EXT4 filesystem utilizes JBD2 journaling and an error occurs, the
journaling will be aborted first and the error number will be recorded
into JBD2 superblock and, finally, the system will enter into the
panic state in "errors=panic" option. But, in the rare case, this
sequence is little twisted like the below figure and it will happen
that the system enters into panic state, which means the system reset
in mobile environment, before completion of recording an error in the
journal superblock. In this case, e2fsck cannot recognize that the
filesystem failure occurred in the previous run and the corruption
wouldn't be fixed.Task A Task B
ext4_handle_error()
-> jbd2_journal_abort()
-> __journal_abort_soft()
-> __jbd2_journal_abort_hard()
| -> journal->j_flags |= JBD2_ABORT;
|
| __ext4_abort()
| -> jbd2_journal_abort()
| | -> __journal_abort_soft()
| | -> if (journal->j_flags & JBD2_ABORT)
| | return;
| -> panic()
|
-> jbd2_journal_update_sb_errno()Tested-by: Hobin Woo
Signed-off-by: Daeho Jeong
Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org
18 Oct, 2015
10 commits
-
"group" is the group where the backup will be placed, and is
initialized to zero in the declaration. This meant that backups for
meta_bg descriptors were erroneously written to the backup block group
descriptors in groups 1 and (desc_per_block-1).Reproduction information:
mke2fs -Fq -t ext4 -b 1024 -O ^resize_inode /tmp/foo.img 16G
truncate -s 24G /tmp/foo.img
losetup /dev/loop0 /tmp/foo.img
mount /dev/loop0 /mnt
resize2fs /dev/loop0
umount /dev/loop0
dd if=/dev/zero of=/dev/loop0 bs=1024 count=2
e2fsck -fy /dev/loop0
losetup -d /dev/loop0Signed-off-by: Andy Leiserson
Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org -
There is a use-after-free possibility in __ext4_journal_stop() in the
case that we free the handle in the first jbd2_journal_stop() because
we're referencing handle->h_err afterwards. This was introduced in
9705acd63b125dee8b15c705216d7186daea4625 and it is wrong. Fix it by
storing the handle->h_err value beforehand and avoid referencing
potentially freed handle.Fixes: 9705acd63b125dee8b15c705216d7186daea4625
Signed-off-by: Lukas Czerner
Reviewed-by: Andreas Dilger
Cc: stable@vger.kernel.org -
Unlike comments and expectation of callers journal_clean_one_cp_list()
returned 1 not only if it freed the transaction but also if it freed
some buffers in the transaction. That could make
__jbd2_journal_clean_checkpoint_list() skip processing
t_checkpoint_io_list and continue with processing the next transaction.
This is mostly a cosmetic issue since the only result is we can
sometimes free less memory than we could. But it's still worth fixing.
Fix journal_clean_one_cp_list() to return 1 only if the transaction was
really freed.Fixes: 50849db32a9f529235a84bcc84a6b8e631b1d0ec
Signed-off-by: Jan Kara
Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org -
When you repeatly execute xfstest generic/269 with bigalloc_1k option
enabled using the below command:"./kvm-xfstests -c bigalloc_1k -m nodelalloc -C 1000 generic/269"
you can easily see the below bug message.
"JBD2 unexpected failure: jbd2_journal_revoke: !buffer_revoked(bh);"
This means that an already revoked buffer is erroneously revoked again
and it is caused by doing revoke for the buffer at the wrong position
in ext4_free_blocks(). We need to re-position the buffer revoke
procedure for an unspecified buffer after checking the cluster boundary
for bigalloc option. If not, some part of the cluster can be doubly
revoked.Signed-off-by: Daeho Jeong
-
Make the bitmap reaading routines return real error codes (EIO,
EFSCORRUPTED, EFSBADCRC) which can then be reflected back to
userspace for more precise diagnosis work.In particular, this means that mballoc no longer claims that we're out
of memory if the block bitmaps become corrupt.Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
Create separate predicate functions to test/set/clear feature flags,
thereby replacing the wordy old macros. Furthermore, clean out the
places where we open-coded feature tests.Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
Create separate predicate functions to test/set/clear feature flags,
thereby replacing the wordy old macros. Furthermore, clean out the
places where we open-coded feature tests.Signed-off-by: Darrick J. Wong
-
Instead of overloading EIO for CRC errors and corrupt structures,
return the same error codes that XFS returns for the same issues.Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
Allow the filesystem to store the metadata checksum seed in the
superblock and add an incompat feature to say that we're using it.
This enables tune2fs to change the UUID on a mounted metadata_csum
FS without having to (racy!) rewrite all disk metadata.Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
Signed-off-by: Theodore Ts'o
15 Oct, 2015
3 commits
-
Prevent clean ext3 filesystems from mounting by default with the ext2
driver (with no journal!) by putting ext4 ahead of ext2 in the default
probe order. This will have the effect of mounting ext2 filesystems
with ext4.ko by default, which is a safer failure than hoping the user
notices that their journalled ext3 is now running without a journal!Users who require ext2.ko for ext2 can either disable ext4.ko or
explicitly request ext2 via "mount -t ext2" or "rootfstype=ext2".Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
Change the journal's checksum functions to gate on whether or not the
crc32c driver is loaded, and gate the loading on the superblock bits.
This prevents a journal crash if someone loads a journal in no-csum
mode and then randomizes the superblock, thus flipping on the feature
bits.Tested-By: Nikolay Borisov
Reported-by: Nikolay Borisov
Signed-off-by: Darrick J. Wong
Signed-off-by: Theodore Ts'o -
If there is a error while copying data from userspace into the page
cache during a write(2) system call, in data=journal mode, in
ext4_journalled_write_end() were using page_zero_new_buffers() from
fs/buffer.c. Unfortunately, this sets the buffer dirty flag, which is
no good if journalling is enabled. This is a long-standing bug that
goes back for years and years in ext3, but a combination of (a)
data=journal not being very common, (b) in many case it only results
in a warning message. and (c) only very rarely causes the kernel hang,
means that we only really noticed this as a problem when commit
998ef75ddb caused this failure to happen frequently enough to cause
generic/208 to fail when run in data=journal mode.The fix is to have our own version of this function that doesn't call
mark_dirty_buffer(), since we will end up calling
ext4_handle_dirty_metadata() on the buffer head(s) in questions very
shortly afterwards in ext4_journalled_write_end().Thanks to Dave Hansen and Linus Torvalds for helping to identify the
root cause of the problem.Signed-off-by: Theodore Ts'o
Reviewed-by: Jan Kara
03 Oct, 2015
5 commits
-
Fix multiple bugs in ext4_encrypted_zeroout(), including one that
could cause us to write an encrypted zero page to the wrong location
on disk, potentially causing data and file system corruption.
Fortunately, this tends to only show up in stress tests, but even with
these fixes, we are seeing some test failures with generic/127 --- but
these are now caused by data failures instead of metadata corruption.Since ext4_encrypted_zeroout() is only used for some optimizations to
keep the extent tree from being too fragmented, and
ext4_encrypted_zeroout() itself isn't all that optimized from a time
or IOPS perspective, disable the extent tree optimization for
encrypted inodes for now. This prevents the data corruption issues
reported by generic/127 until we can figure out what's going wrong.Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org -
Buggy (or hostile) userspace should not be able to cause the kernel to
crash.Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org -
Since ext4_page_crypto() doesn't need an encryption context (at least
not any more), this allows us to simplify a number function signature
and also allows us to avoid needing to allocate a context in
ext4_block_write_begin(). It also means we no longer need a separate
ext4_decrypt_one() function.Signed-off-by: Theodore Ts'o
-
In cases where the file system block size is the same as the page
size, and ext4_writepage() is asked to write out a page which is
either has the unwritten bit set in the extent tree, or which does not
yet have a block assigned due to delayed allocation, we can bail out
early and, unlocking the page earlier and avoiding a round trip
through ext4_bio_write_page() with the attendant calls to
set_page_writeback() and redirty_page_for_writeback().Signed-off-by: Theodore Ts'o
-
There are times when ext4_bio_write_page() is called even though we
don't actually need to do any I/O. This happens when ext4_writepage()
gets called by the jbd2 commit path when an inode needs to force its
pages written out in order to provide data=ordered guarantees --- and
a page is backed by an unwritten (e.g., uninitialized) block on disk,
or if delayed allocation means the page's backing store hasn't been
allocated yet. In that case, we need to skip the call to
ext4_encrypt_page(), since in addition to wasting CPU, it leads to a
bounce page and an ext4 crypto context getting leaked.Signed-off-by: Theodore Ts'o
Cc: stable@vger.kernel.org
24 Sep, 2015
3 commits
-
This allows us to refactor the procfs code, which saves a bit of
compiled space. More importantly it isolates most of the procfs
support code into a single file, so it's easier to #ifdef it out if
the proc file system has been disabled.Signed-off-by: Theodore Ts'o
-
Make the code more easily extensible as well as taking up less
compiled space.Signed-off-by: Theodore Ts'o
-
Also statically allocate the ext4_kset and ext4_feat objects, since we
only need exactly one of each, and it's simpler and less code if we
drop the dynamic allocation and deallocation when it's not needed.Signed-off-by: Theodore Ts'o
21 Sep, 2015
1 commit
20 Sep, 2015
9 commits
-
Pull ARM fixes from Russell King:
"Three fixes and a resulting cleanup for -rc2:- Andre Przywara reported that he was seeing a warning with the new
cast inside DMA_ERROR_CODE's definition, and fixed the incorrect
use.- Doug Anderson noticed that kgdb causes a "scheduling while atomic"
bug.- OMAP5 folk noticed that their Thumb-2 compiled X servers crashed
when enabling support to cover ARMv6 CPUs due to a kernel bug
leaking some conditional context into the signal handler"* 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm:
ARM: 8425/1: kgdb: Don't try to stop the machine when setting breakpoints
ARM: 8437/1: dma-mapping: fix build warning with new DMA_ERROR_CODE definition
ARM: get rid of needless #if in signal handling code
ARM: fix Thumb2 signal handling when ARMv6 is enabled -
…/git/shuah/linux-kselftest
Pull kselftest fixes from Shuah Khan:
"This update contains 7 fixes for problems ranging from build failurs
to incorrect error reporting"* tag 'linux-kselftest-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
selftests: exec: revert to default emit rule
selftests: change install command to rsync
selftests: mqueue: simplify the Makefile
selftests: mqueue: allow extra cflags
selftests: rename jump label to static_keys
selftests/seccomp: add support for s390
seltests/zram: fix syntax error -
Pull power management and ACPI updates from Rafael Wysocki:
"Included are: a somewhat late devfreq update which however is mostly
fixes and cleanups with one new thing only (the PPMUv2 support on
Exynos5433), an ACPI cpufreq driver fixup and two ACPI core cleanups
related to preprocessor directives.Specifics:
- Fix a memory allocation size in the devfreq core (Xiaolong Ye).
- Fix a mistake in the exynos-ppmu DT binding (Javier Martinez
Canillas).- Add support for PPMUv2 ((Platform Performance Monitoring Unit
version 2.0) on the Exynos5433 SoCs (Chanwoo Choi).- Fix a type casting bug in the Exynos PPMU code (MyungJoo Ham).
- Assorted devfreq code cleanups and optimizations (Javi Merino,
MyungJoo Ham, Viresh Kumar).- Fix up the ACPI cpufreq driver to use a more lightweight way to get
to its private data in the ->get() callback (Rafael J Wysocki).- Fix a CONFIG_ prefix bug in one of the ACPI drivers and make the
ACPI subsystem use IS_ENABLED() instead of #ifdefs in function
bodies (Sudeep Holla)"* tag 'pm+acpi-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
cpufreq: acpi-cpufreq: Use cpufreq_cpu_get_raw() in ->get()
ACPI: Eliminate CONFIG_.*{, _MODULE} #ifdef in favor of IS_ENABLED()
ACPI: int340x_thermal: add missing CONFIG_ prefix
PM / devfreq: Fix incorrect type issue.
PM / devfreq: tegra: Update governor to use devfreq_update_stats()
PM / devfreq: comments for get_dev_status usage updated
PM / devfreq: drop comment about thermal setting max_freq
PM / devfreq: cache the last call to get_dev_status()
PM / devfreq: Drop unlikely before IS_ERR(_OR_NULL)
PM / devfreq: exynos-ppmu: bit-wise operation bugfix.
PM / devfreq: exynos-ppmu: Update documentation to support PPMUv2
PM / devfreq: exynos-ppmu: Add the support of PPMUv2 for Exynos5433
PM / devfreq: event: Remove incorrect property in exynos-ppmu DT binding -
Pull clk fixes from Stephen Boyd:
"A few driver fixes for tegra, rockchip, and st SoCs and a two-liner in
the framework to avoid oops when get_parent ops return out of range
values on tegra platforms"* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
drivers: clk: st: Rename st_pll3200c32_407_c0_x into st_pll3200c32_cx_x
clk: check for invalid parent index of orphans in __clk_init()
clk: tegra: dfll: Properly protect OPP list
clk: rockchip: add critical clock for rk3368 -
…git/j.anaszewski/linux-leds
Pull LED fixes from Jacek Anaszewski:
- fix module autoload for six OF platform drivers (aat1290, bcm6328,
bcm6358, ktd2692, max77693, ns2)
- aat1290: add missing static modifier
- ipaq-micro: add missing LEDS_CLASS dependency
- lp55xx: correct Kconfig dependecy for f/w user helper* tag 'led-fixes-for-v4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds:
leds:lp55xx: Correct Kconfig dependency for f/w user helper
leds: leds-ipaq-micro: Add LEDS_CLASS dependency
leds: aat1290: add 'static' modifier to init_mm_current_scale
leds: leds-ns2: Fix module autoload for OF platform driver
leds: max77693: Fix module autoload for OF platform driver
leds: ktd2692: Fix module autoload for OF platform driver
leds: bcm6358: Fix module autoload for OF platform driver
leds: bcm6328: Fix module autoload for OF platform driver
leds: aat1290: Fix module autoload for OF platform driver -
Pull rdma fixes from Doug Ledford:
"The new hfi1 driver in staging/rdma has had a number of fixup patches
since being added to the tree. This is the first batch of those fixes"* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma:
IB/hfi: Properly set permissions for user device files
IB/hfi1: mask vs shift confusion
IB/hfi1: clean up some defines
IB/hfi1: info leak in get_ctxt_info()
IB/hfi1: fix a locking bug
IB/hfi1: checking for NULL instead of IS_ERR
IB/hfi1: fix sdma_descq_cnt parameter parsing
IB/hfi1: fix copy_to/from_user() error handling
IB/hfi1: fix pstateinfo from returning improperly byteswapped value -
Pull libnvdimm fixes from Dan Williams:
- a boot regression (since v4.2) fix for some ARM configurations from
Tyler- regression (since v4.1) fixes for mkfs.xfs on a DAX enabled device
from Jeff. These are tagged for -stable.- a pair of locking fixes from Axel that are hidden from lockdep since
they involve device_lock(). The "btt" one is tagged for -stable, the
other only applies to the new "pfn" mechanism in v4.3.- a fix for the pmem ->rw_page() path to use wmb_pmem() from Ross.
* 'libnvdimm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm:
mm: fix type cast in __pfn_to_phys()
pmem: add proper fencing to pmem_rw_page()
libnvdimm: pfn_devs: Fix locking in namespace_store
libnvdimm: btt_devs: Fix locking in namespace_store
blockdev: don't set S_DAX for misaligned partitions
dax: fix O_DIRECT I/O to the last block of a blockdev -
Pull block updates from Jens Axboe:
"This is a bit bigger than it should be, but I could (did) not want to
send it off last week due to both wanting extra testing, and expecting
a fix for the bounce regression as well. In any case, this contains:- Fix for the blk-merge.c compilation warning on gcc 5.x from me.
- A set of back/front SG gap merge fixes, from me and from Sagi.
This ensures that we honor SG gapping for integrity payloads as
well.- Two small fixes for null_blk from Matias, fixing a leak and a
capacity propagation issue.- A blkcg fix from Tejun, fixing a NULL dereference.
- A fast clone optimization from Ming, fixing a performance
regression since the arbitrarily sized bio's were introduced.- Also from Ming, a regression fix for bouncing IOs"
* 'for-linus' of git://git.kernel.dk/linux-block:
block: fix bounce_end_io
block: blk-merge: fast-clone bio when splitting rw bios
block: blkg_destroy_all() should clear q->root_blkg and ->root_rl.blkg
block: Copy a user iovec if it includes gaps
block: Refuse adding appending a gapped integrity page to a bio
block: Refuse request/bio merges with gaps in the integrity payload
block: Check for gaps on front and back merges
null_blk: fix wrong capacity when bs is not 512 bytes
null_blk: fix memory leak on cleanup
block: fix bogus compiler warnings in blk-merge.c -
Commit 505a666ee3fc ("writeback: plug writeback in wb_writeback() and
writeback_inodes_wb()") has us holding a plug during writeback_sb_inodes,
which increases the merge rate when relatively contiguous small files
are written by the filesystem. It helps both on flash and spindles.For an fs_mark workload creating 4K files in parallel across 8 drives,
this commit improves performance ~9% more by unplugging before calling
cond_resched(). cond_resched() doesn't trigger an implicit unplug, so
explicitly getting the IO down to the device before scheduling reduces
latencies for anyone waiting on clean pages.It also cuts down on how often we use kblockd to unplug, which means
less work bouncing from one workqueue to another.Many more details about how we got here:
https://lkml.org/lkml/2015/9/11/570
Signed-off-by: Chris Mason
Signed-off-by: Linus Torvalds
19 Sep, 2015
2 commits
-
The various definitions of __pfn_to_phys() have been consolidated to
use a generic macro in include/asm-generic/memory_model.h. This hit
mainline in the form of 012dcef3f058 "mm: move __phys_to_pfn and
__pfn_to_phys to asm/generic/memory_model.h". When the generic macro
was implemented the type cast to phys_addr_t was dropped which caused
boot regressions on ARM platforms with more than 4GB of memory and
LPAE enabled.It was suggested to use PFN_PHYS() defined in include/linux/pfn.h
as provides the correct logic and avoids further duplication.Reported-by: kernelci.org bot
Suggested-by: Dan Williams
Signed-off-by: Tyler Baker
Signed-off-by: Dan Williams -
* acpi-bus:
ACPI: Eliminate CONFIG_.*{, _MODULE} #ifdef in favor of IS_ENABLED()
ACPI: int340x_thermal: add missing CONFIG_ prefix
