18 Nov, 2019
12 commits
-
Replace indirect calls to free_workspace by switch and calls to the
specific callbacks. This is mainly to get rid of the indirection due to
spectre vulnerability mitigations.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Replace indirect calls to alloc_workspace by switch and calls to the
specific callbacks. This is mainly to get rid of the indirection due to
spectre vulnerability mitigations.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Similar to get_workspace, majority of the callbacks is trivial, we don't
gain anything by the indirection, so replace them by a switch function.
Trivial callback implementations use the helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Majority of the callbacks is trivial, we don't gain anything by the
indirection, so replace them by a switch function.ZLIB needs to adjust level in the callback and ZSTD workspace management
is complex, the rest is call to the helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
The indirect calls will be replaced by a switch in compression.c.
(Switch is faster than indirect calls with when Spectre mitigations are
enabled).Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Replace loop calling to all algos with a list of direct calls to the
cleanup manager callback. When that becomes trivial it is replaced by
direct call to the helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
With the access to the workspace structures, we can look it up together
with the compression ops inside the workspace manager cleanup helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Replace loop calling to all algos with a list of direct calls to the
init manager callback. When that becomes trivial it is replaced by
direct call to the helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
With the access to the workspace structures, we can look it up together
with the compression ops inside the workspace manager init helper.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
There's a lot of indirection when the generic code calls into
algo-specific callbacks to reach the private workspace manager structure
and back to the generic code.To simplify that, export the workspace manager for heuristic, LZO and
ZLIB, while ZSTD is going to use it's own manager.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
The indirect calls bring some overhead due to spectre vulnerability
mitigations. The number of cases is small and below the threshold
(10-20) where indirect call would be better.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba -
Export compress_pages, decompress_bio and decompress callbacks for all
compression algos. The indirect calls will be replaced by a switch.Reviewed-by: Johannes Thumshirn
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba
09 Sep, 2019
2 commits
-
The set_level callbacks do not do anything special and can be replaced
by a helper that uses the levels defined in the tables.Reviewed-by: Nikolay Borisov
Reviewed-by: Anand Jain
Signed-off-by: David Sterba -
The maximum and default levels do not change and can be defined
directly. The set_level callback was a temporary solution and will be
removed.Reviewed-by: Nikolay Borisov
Reviewed-by: Anand Jain
Signed-off-by: David Sterba
25 Feb, 2019
3 commits
-
Currently, the only user of set_level() is zlib which sets an internal
workspace parameter. As level is now plumbed into get_workspace(), this
can be handled there rather than separately.This repurposes set_level() to bound the level passed in so it can be
used when setting the mounts compression level and as well as verifying
the level before getting a workspace. The other benefit is this divides
the meaning of compress(0) and get_workspace(0). The former means we
want to use the default compression level of the compression type. The
latter means we can use any workspace available.Signed-off-by: Dennis Zhou
Reviewed-by: David Sterba
Signed-off-by: David Sterba -
Zlib compression supports multiple levels, but doesn't require changing
in how a workspace itself is created and managed. Zstd introduces a
different memory requirement such that higher levels of compression
require more memory.This requires changes in how the alloc()/get() methods work for zstd.
This pach plumbs compression level through the interface as a parameter
in preparation for zstd compression levels. This gives the compression
types opportunity to create/manage based on the compression level.Reviewed-by: Nikolay Borisov
Reviewed-by: Josef Bacik
Signed-off-by: Dennis Zhou
Reviewed-by: David Sterba
Signed-off-by: David Sterba -
The previous patch added generic helpers for get_workspace() and
put_workspace(). Now, we can migrate ownership of the workspace_manager
to be in the compression type code as the compression code itself
doesn't care beyond being able to get a workspace. The init/cleanup and
get/put methods are abstracted so each compression algorithm can decide
how they want to manage their workspaces.Reviewed-by: Josef Bacik
Signed-off-by: Dennis Zhou
Reviewed-by: David Sterba
Signed-off-by: David Sterba
17 Dec, 2018
1 commit
-
The typos accumulate over time so once in a while time they get fixed in
a large patch.Signed-off-by: Andrea Gelmini
Reviewed-by: David Sterba
Signed-off-by: David Sterba
30 May, 2018
3 commits
-
For inlined extent, we only have one segment, thus less things to check.
And further more, inlined extent always has the csum in its leaf header,
it's less probable to have corrupted data.Anyway, still check header and segment header.
Signed-off-by: Qu Wenruo
Reviewed-by: David Sterba
Signed-off-by: David Sterba -
James Harvey reported that some corrupted compressed extent data can
lead to various kernel memory corruption.Such corrupted extent data belongs to inode with NODATASUM flags, thus
data csum won't help us detecting such bug.If lucky enough, KASAN could catch it like:
BUG: KASAN: slab-out-of-bounds in lzo_decompress_bio+0x384/0x7a0 [btrfs]
Write of size 4096 at addr ffff8800606cb0f8 by task kworker/u16:0/2338CPU: 3 PID: 2338 Comm: kworker/u16:0 Tainted: G O 4.17.0-rc5-custom+ #50
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
Workqueue: btrfs-endio btrfs_endio_helper [btrfs]
Call Trace:
dump_stack+0xc2/0x16b
print_address_description+0x6a/0x270
kasan_report+0x260/0x380
memcpy+0x34/0x50
lzo_decompress_bio+0x384/0x7a0 [btrfs]
end_compressed_bio_read+0x99f/0x10b0 [btrfs]
bio_endio+0x32e/0x640
normal_work_helper+0x15a/0xea0 [btrfs]
process_one_work+0x7e3/0x1470
worker_thread+0x1b0/0x1170
kthread+0x2db/0x390
ret_from_fork+0x22/0x40
...The offending compressed data has the following info:
Header: length 32768 (looks completely valid)
Segment 0 Header: length 3472882419 (obviously out of bounds)Then when handling segment 0, since it's over the current page, we need
the copy the compressed data to temporary buffer in workspace, then such
large size would trigger out-of-bounds memory access, screwing up the
whole kernel.Fix it by adding extra checks on header and segment headers to ensure we
won't access out-of-bounds, and even checks the decompressed data won't
be out-of-bounds.Reported-by: James Harvey
Signed-off-by: Qu Wenruo
Reviewed-by: Misono Tomohiro
Reviewed-by: David Sterba
[ updated comments ]
Signed-off-by: David Sterba -
Although it's not that complex, but such comment could still save
several minutes for newer reader/reviewer instead of inferring that from
the code.Signed-off-by: Qu Wenruo
Reviewed-by: David Sterba
[ minor wording updates ]
Signed-off-by: David Sterba
12 Apr, 2018
1 commit
-
Remove GPL boilerplate text (long, short, one-line) and keep the rest,
ie. personal, company or original source copyright statements. Add the
SPDX header.Signed-off-by: David Sterba
31 Mar, 2018
1 commit
-
Added already unused in a6fa6fae40ec ("btrfs: Add lzo compression
support").Signed-off-by: Nikolay Borisov
Signed-off-by: David Sterba
02 Nov, 2017
1 commit
-
Preliminary support for setting compression level for zlib, the
following works:$ mount -o compess=zlib # default
$ mount -o compess=zlib0 # same
$ mount -o compess=zlib9 # level 9, slower sync, less data
$ mount -o compess=zlib1 # level 1, faster sync, more data
$ mount -o remount,compress=zlib3 # level set by remountThe compress-force works the same as compress'. The level is visible in
the same format in /proc/mounts. Level set via file property does not
work yet.Required patch: "btrfs: prepare for extensions in compression options"
Signed-off-by: David Sterba
20 Jun, 2017
5 commits
-
The compression workspace buffers are larger than a page so we use
vmalloc, unconditionally. This is not always necessary as there might be
contiguous memory available.Let's use the kvmalloc helpers that will try kmalloc first and fallback
to vmalloc. For that they require GFP_KERNEL flags. As we now have the
alloc_workspace calls protected by memalloc_nofs in the critical
contexts, we can safely use GFP_KERNEL.Signed-off-by: David Sterba
-
As alloc_workspace is now protected by memalloc_nofs where needed,
we can switch the kmalloc to use GFP_KERNEL.Reviewed-by: Anand Jain
Signed-off-by: David Sterba -
Logic already skips if compression makes data bigger, let's sync lzo
with zlib and also return error if compressed size is equal to
input size.Signed-off-by: Timofey Titovets
Reviewed-by: David Sterba
[ update changelog ]
Signed-off-by: David Sterba -
Fix copy paste typo in debug message for lzo.c, lzo is not deflate.
Signed-off-by: Timofey Titovets
Signed-off-by: David Sterba -
struct compressed_bio pointer can be used instead.
Signed-off-by: Anand Jain
Reviewed-by: Nikolay Borisov
Signed-off-by: David Sterba
28 Feb, 2017
4 commits
-
The value of max_out can be calculated from the parameters passed to the
compressors, which is number of pages and the page size, and we don't
have to needlessly pass it around.Signed-off-by: David Sterba
-
The parameter saying how many pages can be allocated at maximum can be
merged with the output page counter, to save some stack space. The
compression implementation will sink the parameter to a local variable
so everything works as before.The nr_pages variables can also be simply merged in compress_file_range
into one.Signed-off-by: David Sterba
-
The length parameter is basically duplicated for input and output in the
top level caller of the compress_pages chain. We can simply use one
variable for that and reduce stack consumption. The compression
implementation will sink the parameter to a local variable so everything
works as before.Signed-off-by: David Sterba
-
Signed-off-by: David Sterba
30 Nov, 2016
1 commit
-
Pass the full bio to the decompression routines and use bio iterators
to iterate over the data in the bio.Signed-off-by: Christoph Hellwig
Signed-off-by: David Sterba
27 Sep, 2016
1 commit
-
This patch converts printk(KERN_* style messages to use the pr_* versions.
One side effect is that anything that was KERN_DEBUG is now automatically
a dynamic debug message.Signed-off-by: Jeff Mahoney
Signed-off-by: David Sterba
05 Apr, 2016
1 commit
-
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.Let's stop pretending that pages in page cache are special. They are
not.The changes are pretty straight-forward:
- << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> ;
- >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> ;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)Signed-off-by: Kirill A. Shutemov
Acked-by: Michal Hocko
Signed-off-by: Linus Torvalds
17 Feb, 2015
1 commit
-
There are some op tables that can be easily made const, similarly the
sysfs feature and raid tables. This is motivated by PaX CONSTIFY plugin.Signed-off-by: David Sterba
01 Dec, 2014
1 commit
-
Don Bailey noticed that our page zeroing for compression at end-io time
isn't complete. This reworks a patch from Linus to push the zeroing
into the zlib and lzo specific functions instead of trying to handle the
corners inside btrfs_decompress_buf2pageSigned-off-by: Chris Mason
Reviewed-by: Josef Bacik
Reported-by: Don A. Bailey
cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds
18 Sep, 2014
1 commit
-
The form
(value + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT
is equivalent to
(value + PAGE_CACHE_SIZE - 1) / PAGE_CACHE_SIZE
The rest is a simple subsitution, no difference in the generated
assembly code.Signed-off-by: David Sterba
Signed-off-by: Chris Mason
10 Jun, 2014
1 commit
-
The compression layer seems to have been built to return -1 and have
callers make up errors that make sense. This isn't great because there
are different errors that originate down in the compression layer.Let's return real negative errnos from the compression layer so that
callers can pass on the error without having to guess what happened.
ENOMEM for allocation failure, E2BIG when compression exceeds the
uncompressed input, and EIO for everything else.This helps a future path return errors from btrfs_decompress().
Signed-off-by: Zach Brown
Signed-off-by: Chris Mason
