18 Jan, 2020
1 commit
-
This patch tries to support compression in f2fs.
- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.extCompress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+Changelog:
20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.20190402
- don't preallocate blocks for compressed file.- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDROne cluster contain 4 blocks
before overwrite after overwrite
- VVVV -> CVNN
- CVNN -> VVVV- CVNN -> CVNN
- CVNN -> CVVV- CVVV -> CVNN
- CVVV -> CVVV20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.20191101
- apply fixes from Jaegeuk20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity20191216
- apply fixes from Jaegeuk20200117
- fix to avoid NULL pointer dereference[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocksReported-by:
Signed-off-by: Chao Yu
Signed-off-by: Jaegeuk Kim
13 Aug, 2019
1 commit
-
Add fs-verity support to f2fs. fs-verity is a filesystem feature that
enables transparent integrity protection and authentication of read-only
files. It uses a dm-verity like mechanism at the file level: a Merkle
tree is used to verify any block in the file in log(filesize) time. It
is implemented mainly by helper functions in fs/verity/. See
Documentation/filesystems/fsverity.rst for the full documentation.The f2fs support for fs-verity consists of:
- Adding a filesystem feature flag and an inode flag for fs-verity.
- Implementing the fsverity_operations to support enabling verity on an
inode and reading/writing the verity metadata.- Updating ->readpages() to verify data as it's read from verity files
and to support reading verity metadata pages.- Updating ->write_begin(), ->write_end(), and ->writepages() to support
writing verity metadata pages.- Calling the fs-verity hooks for ->open(), ->setattr(), and ->ioctl().
Like ext4, f2fs stores the verity metadata (Merkle tree and
fsverity_descriptor) past the end of the file, starting at the first 64K
boundary beyond i_size. This approach works because (a) verity files
are readonly, and (b) pages fully beyond i_size aren't visible to
userspace but can be read/written internally by f2fs with only some
relatively small changes to f2fs. Extended attributes cannot be used
because (a) f2fs limits the total size of an inode's xattr entries to
4096 bytes, which wouldn't be enough for even a single Merkle tree
block, and (b) f2fs encryption doesn't encrypt xattrs, yet the verity
metadata *must* be encrypted when the file is because it contains hashes
of the plaintext data.Acked-by: Jaegeuk Kim
Acked-by: Chao Yu
Signed-off-by: Eric Biggers
02 Nov, 2017
1 commit
-
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.By default all files without license information are under the default
license of the kernel, which is GPL version 2.Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if
Reviewed-by: Philippe Ombredanne
Reviewed-by: Thomas Gleixner
Signed-off-by: Greg Kroah-Hartman
04 Jul, 2017
1 commit
-
Codes related to sysfs and procfs are dispersive and mixed with sb
related codes, but actually these codes are independent from others,
so split them from super.c, and reorgnize and manger them in sysfs.c.Signed-off-by: Chao Yu
Signed-off-by: Jaegeuk Kim
18 Mar, 2016
1 commit
-
This patch adds the renamed functions moved from the f2fs crypto files.
1. definitions for per-file encryption used by ext4 and f2fs.
2. crypto.c for encrypt/decrypt functions
a. IO preparation:
- fscrypt_get_ctx / fscrypt_release_ctx
b. before IOs:
- fscrypt_encrypt_page
- fscrypt_decrypt_page
- fscrypt_zeroout_range
c. after IOs:
- fscrypt_decrypt_bio_pages
- fscrypt_pullback_bio_page
- fscrypt_restore_control_page3. policy.c supporting context management.
a. For ioctls:
- fscrypt_process_policy
- fscrypt_get_policy
b. For context permission
- fscrypt_has_permitted_context
- fscrypt_inherit_context4. keyinfo.c to handle permissions
- fscrypt_get_encryption_info
- fscrypt_free_encryption_info5. fname.c to support filename encryption
a. general wrapper functions
- fscrypt_fname_disk_to_usr
- fscrypt_fname_usr_to_disk
- fscrypt_setup_filename
- fscrypt_free_filenameb. specific filename handling functions
- fscrypt_fname_alloc_buffer
- fscrypt_fname_free_buffer6. Makefile and Kconfig
Cc: Al Viro
Signed-off-by: Michael Halcrow
Signed-off-by: Ildar Muslukhov
Signed-off-by: Uday Savagaonkar
Signed-off-by: Theodore Ts'o
Signed-off-by: Arnd Bergmann
Signed-off-by: Jaegeuk Kim
05 Aug, 2015
2 commits
-
This patch moves extent cache related code from data.c into extent_cache.c
since extent cache is independent feature, and its codes are not relate to
others in data.c, it's better for us to maintain them in separated place.There is no functionality change, but several small coding style fixes
including:
* rename __drop_largest_extent to f2fs_drop_largest_extent for exporting;
* rename misspelled word 'untill' to 'until';
* remove unneeded 'return' in the end of f2fs_destroy_extent_tree().Signed-off-by: Chao Yu
Signed-off-by: Jaegeuk Kim -
This patch introduces a shrinker targeting to reduce memory footprint consumed
by a number of in-memory f2fs data structures.In addition, it newly adds:
- sbi->umount_mutex to avoid data races on shrinker and put_super
- sbi->shruinker_run_no to not revisit objectsNote that the basic implementation was copied from fs/ubifs/shrinker.c
Reviewed-by: Chao Yu
Signed-off-by: Jaegeuk Kim
29 May, 2015
4 commits
-
This patch adds filename encryption infra.
Most of codes are copied from ext4 part, but changed to adjust f2fs
directory structure.Signed-off-by: Uday Savagaonkar
Signed-off-by: Ildar Muslukhov
Signed-off-by: Michael Halcrow
Signed-off-by: Theodore Ts'o
Signed-off-by: Jaegeuk Kim -
This patch copies from encrypt_key.c in ext4, and modifies for f2fs.
Use GFP_NOFS, since _f2fs_get_encryption_info is called under f2fs_lock_op.
Signed-off-by: Michael Halcrow
Signed-off-by: Ildar Muslukhov
Signed-off-by: Theodore Ts'o
Signed-off-by: Jaegeuk Kim -
Most of parts were copied from ext4, except:
- add f2fs_restore_and_release_control_page which returns control page and
restore control page
- remove ext4_encrypted_zeroout()
- remove sbi->s_file_encryption_mode & sbi->s_dir_encryption_mode
- add f2fs_end_io_crypto_work for mpage_end_ioSigned-off-by: Michael Halcrow
Signed-off-by: Ildar Muslukhov
Signed-off-by: Theodore Ts'o
Signed-off-by: Jaegeuk Kim -
This patch adds encryption policy and password salt support through ioctl
implementation.It adds three ioctls:
F2FS_IOC_SET_ENCRYPTION_POLICY,
F2FS_IOC_GET_ENCRYPTION_POLICY,
F2FS_IOC_GET_ENCRYPTION_PWSALT, which use xattr operations.Note that, these definition and codes are taken from ext4 crypto support.
For f2fs, xattr operations and on-disk flags for superblock and inode were
changed.Signed-off-by: Michael Halcrow
Signed-off-by: Theodore Ts'o
Signed-off-by: Ildar Muslukhov
Signed-off-by: Jaegeuk Kim
10 Jan, 2015
1 commit
-
This patch adds:
o initial trace.c and trace.h with skeleton functions
o Kconfig and Makefile to activate this featureSigned-off-by: Jaegeuk Kim
26 Dec, 2013
1 commit
-
Functions to implement inline data read/write, and move inline data to
normal data block when file size exceeds inline data limitation.Signed-off-by: Huajun Li
Signed-off-by: Haicheng Li
Signed-off-by: Weihong Xu
Signed-off-by: Jaegeuk Kim
11 Dec, 2012
1 commit
-
This adds Makefile and Kconfig for f2fs, and updates Makefile and Kconfig files
in the fs directory.Signed-off-by: Jaegeuk Kim
