Commit 7dd65feb6c603e13eba501c34c662259ab38e70e

Authored by Albin Tonnerre
Committed by Linus Torvalds
1 parent ac4c2a3bbe

lib: add support for LZO-compressed kernels

This patch series adds generic support for creating and extracting
LZO-compressed kernel images, as well as support for using such images on
the x86 and ARM architectures, and support for creating and using
LZO-compressed initrd and initramfs images.

Russell King said:

: Testing on a Cortex A9 model:
: - lzo decompressor is 65% of the time gzip takes to decompress a kernel
: - lzo kernel is 9% larger than a gzip kernel
:
: which I'm happy to say confirms your figures when comparing the two.
:
: However, when comparing your new gzip code to the old gzip code:
: - new is 99% of the size of the old code
: - new takes 42% of the time to decompress than the old code
:
: What this means is that for a proper comparison, the results get even better:
: - lzo is 7.5% larger than the old gzip'd kernel image
: - lzo takes 28% of the time that the old gzip code took
:
: So the expense seems definitely worth the effort.  The only reason I
: can think of ever using gzip would be if you needed the additional
: compression (eg, because you have limited flash to store the image.)
:
: I would argue that the default for ARM should therefore be LZO.

This patch:

The lzo compressor is worse than gzip at compression, but faster at
extraction.  Here are some figures for an ARM board I'm working on:

Uncompressed size: 3.24Mo
gzip  1.61Mo 0.72s
lzo   1.75Mo 0.48s

So for a compression ratio that is still relatively close to gzip, it's
much faster to extract, at least in that case.

This part contains:
 - Makefile routine to support lzo compression
 - Fixes to the existing lzo compressor so that it can be used in
   compressed kernels
 - wrapper around the existing lzo1x_decompress, as it only extracts one
   block at a time, while we need to extract a whole file here
 - config dialog for kernel compression

[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: cleanup]
Signed-off-by: Albin Tonnerre <albin.tonnerre@free-electrons.com>
Tested-by: Wu Zhangjin <wuzhangjin@gmail.com>
Acked-by: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Russell King <rmk@arm.linux.org.uk>
Acked-by: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Showing 5 changed files with 244 additions and 7 deletions Side-by-side Diff

include/linux/decompress/unlzo.h
  1 +#ifndef DECOMPRESS_UNLZO_H
  2 +#define DECOMPRESS_UNLZO_H
  3 +
  4 +int unlzo(unsigned char *inbuf, int len,
  5 + int(*fill)(void*, unsigned int),
  6 + int(*flush)(void*, unsigned int),
  7 + unsigned char *output,
  8 + int *pos,
  9 + void(*error)(char *x));
  10 +#endif
... ... @@ -115,10 +115,13 @@
115 115 config HAVE_KERNEL_LZMA
116 116 bool
117 117  
  118 +config HAVE_KERNEL_LZO
  119 + bool
  120 +
118 121 choice
119 122 prompt "Kernel compression mode"
120 123 default KERNEL_GZIP
121   - depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA
  124 + depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_LZO
122 125 help
123 126 The linux kernel is a kind of self-extracting executable.
124 127 Several compression algorithms are available, which differ
... ... @@ -141,9 +144,8 @@
141 144 bool "Gzip"
142 145 depends on HAVE_KERNEL_GZIP
143 146 help
144   - The old and tried gzip compression. Its compression ratio is
145   - the poorest among the 3 choices; however its speed (both
146   - compression and decompression) is the fastest.
  147 + The old and tried gzip compression. It provides a good balance
  148 + between compression ratio and decompression speed.
147 149  
148 150 config KERNEL_BZIP2
149 151 bool "Bzip2"
... ... @@ -163,6 +165,14 @@
163 165 Its ratio is best, decompression speed is between the other
164 166 two. Compression is slowest. The kernel size is about 33%
165 167 smaller with LZMA in comparison to gzip.
  168 +
  169 +config KERNEL_LZO
  170 + bool "LZO"
  171 + depends on HAVE_KERNEL_LZO
  172 + help
  173 + Its compression ratio is the poorest among the 4. The kernel
  174 + size is about about 10% bigger than gzip; however its speed
  175 + (both compression and decompression) is the fastest.
166 176  
167 177 endchoice
168 178  
lib/decompress_unlzo.c
  1 +/*
  2 + * LZO decompressor for the Linux kernel. Code borrowed from the lzo
  3 + * implementation by Markus Franz Xaver Johannes Oberhumer.
  4 + *
  5 + * Linux kernel adaptation:
  6 + * Copyright (C) 2009
  7 + * Albin Tonnerre, Free Electrons <albin.tonnerre@free-electrons.com>
  8 + *
  9 + * Original code:
  10 + * Copyright (C) 1996-2005 Markus Franz Xaver Johannes Oberhumer
  11 + * All Rights Reserved.
  12 + *
  13 + * lzop and the LZO library are free software; you can redistribute them
  14 + * and/or modify them under the terms of the GNU General Public License as
  15 + * published by the Free Software Foundation; either version 2 of
  16 + * the License, or (at your option) any later version.
  17 + *
  18 + * This program is distributed in the hope that it will be useful,
  19 + * but WITHOUT ANY WARRANTY; without even the implied warranty of
  20 + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  21 + * GNU General Public License for more details.
  22 + *
  23 + * You should have received a copy of the GNU General Public License
  24 + * along with this program; see the file COPYING.
  25 + * If not, write to the Free Software Foundation, Inc.,
  26 + * 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
  27 + *
  28 + * Markus F.X.J. Oberhumer
  29 + * <markus@oberhumer.com>
  30 + * http://www.oberhumer.com/opensource/lzop/
  31 + */
  32 +
  33 +#ifdef STATIC
  34 +#include "lzo/lzo1x_decompress.c"
  35 +#else
  36 +#include <linux/slab.h>
  37 +#include <linux/decompress/unlzo.h>
  38 +#endif
  39 +
  40 +#include <linux/types.h>
  41 +#include <linux/lzo.h>
  42 +#include <linux/decompress/mm.h>
  43 +
  44 +#include <linux/compiler.h>
  45 +#include <asm/unaligned.h>
  46 +
  47 +static const unsigned char lzop_magic[] = {
  48 + 0x89, 0x4c, 0x5a, 0x4f, 0x00, 0x0d, 0x0a, 0x1a, 0x0a };
  49 +
  50 +#define LZO_BLOCK_SIZE (256*1024l)
  51 +#define HEADER_HAS_FILTER 0x00000800L
  52 +
  53 +STATIC inline int INIT parse_header(u8 *input, u8 *skip)
  54 +{
  55 + int l;
  56 + u8 *parse = input;
  57 + u8 level = 0;
  58 + u16 version;
  59 +
  60 + /* read magic: 9 first bits */
  61 + for (l = 0; l < 9; l++) {
  62 + if (*parse++ != lzop_magic[l])
  63 + return 0;
  64 + }
  65 + /* get version (2bytes), skip library version (2),
  66 + * 'need to be extracted' version (2) and
  67 + * method (1) */
  68 + version = get_unaligned_be16(parse);
  69 + parse += 7;
  70 + if (version >= 0x0940)
  71 + level = *parse++;
  72 + if (get_unaligned_be32(parse) & HEADER_HAS_FILTER)
  73 + parse += 8; /* flags + filter info */
  74 + else
  75 + parse += 4; /* flags */
  76 +
  77 + /* skip mode and mtime_low */
  78 + parse += 8;
  79 + if (version >= 0x0940)
  80 + parse += 4; /* skip mtime_high */
  81 +
  82 + l = *parse++;
  83 + /* don't care about the file name, and skip checksum */
  84 + parse += l + 4;
  85 +
  86 + *skip = parse - input;
  87 + return 1;
  88 +}
  89 +
  90 +STATIC inline int INIT unlzo(u8 *input, int in_len,
  91 + int (*fill) (void *, unsigned int),
  92 + int (*flush) (void *, unsigned int),
  93 + u8 *output, int *posp,
  94 + void (*error_fn) (char *x))
  95 +{
  96 + u8 skip = 0, r = 0;
  97 + u32 src_len, dst_len;
  98 + size_t tmp;
  99 + u8 *in_buf, *in_buf_save, *out_buf;
  100 + int obytes_processed = 0;
  101 +
  102 + set_error_fn(error_fn);
  103 +
  104 + if (output) {
  105 + out_buf = output;
  106 + } else if (!flush) {
  107 + error("NULL output pointer and no flush function provided");
  108 + goto exit;
  109 + } else {
  110 + out_buf = malloc(LZO_BLOCK_SIZE);
  111 + if (!out_buf) {
  112 + error("Could not allocate output buffer");
  113 + goto exit;
  114 + }
  115 + }
  116 +
  117 + if (input && fill) {
  118 + error("Both input pointer and fill function provided, don't know what to do");
  119 + goto exit_1;
  120 + } else if (input) {
  121 + in_buf = input;
  122 + } else if (!fill || !posp) {
  123 + error("NULL input pointer and missing position pointer or fill function");
  124 + goto exit_1;
  125 + } else {
  126 + in_buf = malloc(lzo1x_worst_compress(LZO_BLOCK_SIZE));
  127 + if (!in_buf) {
  128 + error("Could not allocate input buffer");
  129 + goto exit_1;
  130 + }
  131 + }
  132 + in_buf_save = in_buf;
  133 +
  134 + if (posp)
  135 + *posp = 0;
  136 +
  137 + if (fill)
  138 + fill(in_buf, lzo1x_worst_compress(LZO_BLOCK_SIZE));
  139 +
  140 + if (!parse_header(input, &skip)) {
  141 + error("invalid header");
  142 + goto exit_2;
  143 + }
  144 + in_buf += skip;
  145 +
  146 + if (posp)
  147 + *posp = skip;
  148 +
  149 + for (;;) {
  150 + /* read uncompressed block size */
  151 + dst_len = get_unaligned_be32(in_buf);
  152 + in_buf += 4;
  153 +
  154 + /* exit if last block */
  155 + if (dst_len == 0) {
  156 + if (posp)
  157 + *posp += 4;
  158 + break;
  159 + }
  160 +
  161 + if (dst_len > LZO_BLOCK_SIZE) {
  162 + error("dest len longer than block size");
  163 + goto exit_2;
  164 + }
  165 +
  166 + /* read compressed block size, and skip block checksum info */
  167 + src_len = get_unaligned_be32(in_buf);
  168 + in_buf += 8;
  169 +
  170 + if (src_len <= 0 || src_len > dst_len) {
  171 + error("file corrupted");
  172 + goto exit_2;
  173 + }
  174 +
  175 + /* decompress */
  176 + tmp = dst_len;
  177 + r = lzo1x_decompress_safe((u8 *) in_buf, src_len,
  178 + out_buf, &tmp);
  179 +
  180 + if (r != LZO_E_OK || dst_len != tmp) {
  181 + error("Compressed data violation");
  182 + goto exit_2;
  183 + }
  184 +
  185 + obytes_processed += dst_len;
  186 + if (flush)
  187 + flush(out_buf, dst_len);
  188 + if (output)
  189 + out_buf += dst_len;
  190 + if (posp)
  191 + *posp += src_len + 12;
  192 + if (fill) {
  193 + in_buf = in_buf_save;
  194 + fill(in_buf, lzo1x_worst_compress(LZO_BLOCK_SIZE));
  195 + } else
  196 + in_buf += src_len;
  197 + }
  198 +
  199 +exit_2:
  200 + if (!input)
  201 + free(in_buf);
  202 +exit_1:
  203 + if (!output)
  204 + free(out_buf);
  205 +exit:
  206 + return obytes_processed;
  207 +}
  208 +
  209 +#define decompress unlzo
lib/lzo/lzo1x_decompress.c
... ... @@ -11,11 +11,13 @@
11 11 * Richard Purdie <rpurdie@openedhand.com>
12 12 */
13 13  
  14 +#ifndef STATIC
14 15 #include <linux/module.h>
15 16 #include <linux/kernel.h>
16   -#include <linux/lzo.h>
17   -#include <asm/byteorder.h>
  17 +#endif
  18 +
18 19 #include <asm/unaligned.h>
  20 +#include <linux/lzo.h>
19 21 #include "lzodefs.h"
20 22  
21 23 #define HAVE_IP(x, ip_end, ip) ((size_t)(ip_end - ip) < (x))
22 24  
... ... @@ -244,9 +246,11 @@
244 246 *out_len = op - out;
245 247 return LZO_E_LOOKBEHIND_OVERRUN;
246 248 }
247   -
  249 +#ifndef STATIC
248 250 EXPORT_SYMBOL_GPL(lzo1x_decompress_safe);
249 251  
250 252 MODULE_LICENSE("GPL");
251 253 MODULE_DESCRIPTION("LZO1X Decompressor");
  254 +
  255 +#endif
scripts/Makefile.lib
... ... @@ -235,4 +235,9 @@
235 235 cmd_lzma = (cat $(filter-out FORCE,$^) | \
236 236 lzma -9 && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
237 237 (rm -f $@ ; false)
  238 +
  239 +quiet_cmd_lzo = LZO $@
  240 +cmd_lzo = (cat $(filter-out FORCE,$^) | \
  241 + lzop -9 && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
  242 + (rm -f $@ ; false)