Commit 538f19526e40ce7a5a296fad6a3121409c890adc

Authored by Wu Zhangjin
Committed by Ralf Baechle
1 parent e6299d2677

MIPS: Tracing: Add dynamic function tracer support

With dynamic function tracer, by default, _mcount is defined as an
"empty" function, it returns directly without any more action . When
enabling it in user-space, it will jump to a real tracing
function(ftrace_caller), and do the real job for us.

Differ from the static function tracer, dynamic function tracer provides
two functions ftrace_make_call()/ftrace_make_nop() to enable/disable the
tracing of some indicated kernel functions(set_ftrace_filter).

In the -v4 version, the implementation of this support is basically the same as
X86 version does: _mcount is implemented as an empty function and ftrace_caller
is implemented as a real tracing function respectively.

But in this version, to support module tracing with the help of
-mlong-calls in arch/mips/Makefile:

MODFLAGS += -mlong-calls.

The stuff becomes a little more complex. We need to cope with two
different type of calling to _mcount.

For the kernel part, the calling to _mcount(result of "objdump -hdr
vmlinux"). is like this:

	108:   03e0082d        move    at,ra
	10c:   0c000000        jal     0 <fpcsr_pending>
                        10c: R_MIPS_26  _mcount
                        10c: R_MIPS_NONE        *ABS*
                        10c: R_MIPS_NONE        *ABS*
	110:   00020021        nop

For the module with -mlong-calls, it looks like this:

	c:	3c030000 	lui	v1,0x0
			c: R_MIPS_HI16	_mcount
			c: R_MIPS_NONE	*ABS*
			c: R_MIPS_NONE	*ABS*
	10:	64630000 	daddiu	v1,v1,0
			10: R_MIPS_LO16	_mcount
			10: R_MIPS_NONE	*ABS*
			10: R_MIPS_NONE	*ABS*
	14:	03e0082d 	move	at,ra
	18:	0060f809 	jalr	v1

In the kernel version, there is only one "_mcount" string for every
kernel function, so, we just need to match this one in mcount_regex of
scripts/recordmcount.pl, but in the module version, we need to choose
one of the two to match. Herein, I choose the first one with
"R_MIPS_HI16 _mcount".

and In the kernel verion, without module tracing support, we just need
to replace "jal _mcount" by "jal ftrace_caller" to do real tracing, and
filter the tracing of some kernel functions via replacing it by a nop
instruction.

but as we have described before, the instruction "jal ftrace_caller" only left
32bit length for the address of ftrace_caller, it will fail when calling from
the module space. so, herein, we must replace something else.

the basic idea is loading the address of ftrace_caller to v1 via changing these
two instructions:

	lui	v1,0x0
	addiu	v1,v1,0

If we want to enable the tracing, we need to replace the above instructions to:

	lui	v1, HI_16BIT_ftrace_caller
	addiu	v1, v1, LOW_16BIT_ftrace_caller

If we want to stop the tracing of the indicated kernel functions, we
just need to replace the "jalr v1" to a nop instruction. but we need to
replace two instructions and encode the above two instructions
oursevles.

Is there a simpler solution? Yes! Here it is, in this version, we put _mcount
and ftrace_caller together, which means the address of _mcount and
ftrace_caller is the same:

_mcount:
ftrace_caller:
	j	ftrace_stub
	 nop

	...(do real tracing here)...

ftrace_stub:
	jr	ra
	 move	ra, at

By default, the kernel functions call _mcount, and then jump to ftrace_stub and
return. and when we want to do real tracing, we just need to remove that "j
ftrace_stub", and it will run through the two "nop" instructions and then do
the real tracing job.

what about filtering job? we just need to do this:

	 lui v1, hi_16bit_of_mcount        <--> b 1f (0x10000004)
	 addiu v1, v1, low_16bit_of_mcount
	 move at, ra
	 jalr v1
	 nop
	 				     1f: (rec->ip + 12)

In linux-mips64, there will be some local symbols, whose name are
prefixed by $L, which need to be filtered. thanks goes to Steven for
writing the mips64-specific function_regex.

In a conclusion, with RISC, things becomes easier with such a "stupid"
trick, RISC is something like K.I.S.S, and also, there are lots of
"simple" tricks in the whole ftrace support, thanks goes to Steven and
the other folks for providing such a wonderful tracing framework!

Signed-off-by: Wu Zhangjin <wuzhangjin@gmail.com>
Cc: Nicholas Mc Guire <der.herr@hofr.at>
Cc: zhangfx@lemote.com
Cc: Wu Zhangjin <wuzhangjin@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/675/
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

Showing 6 changed files with 206 additions and 1 deletions Side-by-side Diff

... ... @@ -7,6 +7,8 @@
7 7 select HAVE_ARCH_KGDB
8 8 select HAVE_FUNCTION_TRACER
9 9 select HAVE_FUNCTION_TRACE_MCOUNT_TEST
  10 + select HAVE_DYNAMIC_FTRACE
  11 + select HAVE_FTRACE_MCOUNT_RECORD
10 12 # Horrible source of confusion. Die, die, die ...
11 13 select EMBEDDED
12 14 select RTC_LIB if !MACH_LOONGSON
arch/mips/include/asm/ftrace.h
... ... @@ -19,6 +19,15 @@
19 19 extern void _mcount(void);
20 20 #define mcount _mcount
21 21  
  22 +#ifdef CONFIG_DYNAMIC_FTRACE
  23 +static inline unsigned long ftrace_call_adjust(unsigned long addr)
  24 +{
  25 + return addr;
  26 +}
  27 +
  28 +struct dyn_arch_ftrace {
  29 +};
  30 +#endif /* CONFIG_DYNAMIC_FTRACE */
22 31 #endif /* __ASSEMBLY__ */
23 32 #endif /* CONFIG_FUNCTION_TRACER */
24 33 #endif /* _ASM_MIPS_FTRACE_H */
arch/mips/kernel/Makefile
... ... @@ -11,6 +11,7 @@
11 11 time.o topology.o traps.o unaligned.o watch.o
12 12  
13 13 ifdef CONFIG_FUNCTION_TRACER
  14 +CFLAGS_REMOVE_ftrace.o = -pg
14 15 CFLAGS_REMOVE_early_printk.o = -pg
15 16 endif
16 17  
... ... @@ -31,7 +32,7 @@
31 32 obj-$(CONFIG_STACKTRACE) += stacktrace.o
32 33 obj-$(CONFIG_MODULES) += mips_ksyms.o module.o
33 34  
34   -obj-$(CONFIG_FUNCTION_TRACER) += mcount.o
  35 +obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o
35 36  
36 37 obj-$(CONFIG_CPU_LOONGSON2) += r4k_fpu.o r4k_switch.o
37 38 obj-$(CONFIG_CPU_MIPS32) += r4k_fpu.o r4k_switch.o
arch/mips/kernel/ftrace.c
  1 +/*
  2 + * Code for replacing ftrace calls with jumps.
  3 + *
  4 + * Copyright (C) 2007-2008 Steven Rostedt <srostedt@redhat.com>
  5 + * Copyright (C) 2009 DSLab, Lanzhou University, China
  6 + * Author: Wu Zhangjin <wuzj@lemote.com>
  7 + *
  8 + * Thanks goes to Steven Rostedt for writing the original x86 version.
  9 + */
  10 +
  11 +#include <linux/uaccess.h>
  12 +#include <linux/init.h>
  13 +#include <linux/ftrace.h>
  14 +
  15 +#include <asm/cacheflush.h>
  16 +
  17 +#ifdef CONFIG_DYNAMIC_FTRACE
  18 +
  19 +#define JAL 0x0c000000 /* jump & link: ip --> ra, jump to target */
  20 +#define ADDR_MASK 0x03ffffff /* op_code|addr : 31...26|25 ....0 */
  21 +#define jump_insn_encode(op_code, addr) \
  22 + ((unsigned int)((op_code) | (((addr) >> 2) & ADDR_MASK)))
  23 +
  24 +static unsigned int ftrace_nop = 0x00000000;
  25 +
  26 +static int ftrace_modify_code(unsigned long ip, unsigned int new_code)
  27 +{
  28 + *(unsigned int *)ip = new_code;
  29 +
  30 + flush_icache_range(ip, ip + 8);
  31 +
  32 + return 0;
  33 +}
  34 +
  35 +static int lui_v1;
  36 +static int jal_mcount;
  37 +
  38 +int ftrace_make_nop(struct module *mod,
  39 + struct dyn_ftrace *rec, unsigned long addr)
  40 +{
  41 + unsigned int new;
  42 + unsigned long ip = rec->ip;
  43 +
  44 + /* We have compiled module with -mlong-calls, but compiled the kernel
  45 + * without it, we need to cope with them respectively. */
  46 + if (ip & 0x40000000) {
  47 + /* record it for ftrace_make_call */
  48 + if (lui_v1 == 0)
  49 + lui_v1 = *(unsigned int *)ip;
  50 +
  51 + /* lui v1, hi_16bit_of_mcount --> b 1f (0x10000004)
  52 + * addiu v1, v1, low_16bit_of_mcount
  53 + * move at, ra
  54 + * jalr v1
  55 + * nop
  56 + * 1f: (ip + 12)
  57 + */
  58 + new = 0x10000004;
  59 + } else {
  60 + /* record/calculate it for ftrace_make_call */
  61 + if (jal_mcount == 0) {
  62 + /* We can record it directly like this:
  63 + * jal_mcount = *(unsigned int *)ip;
  64 + * Herein, jump over the first two nop instructions */
  65 + jal_mcount = jump_insn_encode(JAL, (MCOUNT_ADDR + 8));
  66 + }
  67 +
  68 + /* move at, ra
  69 + * jalr v1 --> nop
  70 + */
  71 + new = ftrace_nop;
  72 + }
  73 + return ftrace_modify_code(ip, new);
  74 +}
  75 +
  76 +static int modified; /* initialized as 0 by default */
  77 +
  78 +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
  79 +{
  80 + unsigned int new;
  81 + unsigned long ip = rec->ip;
  82 +
  83 + /* We just need to remove the "b ftrace_stub" at the fist time! */
  84 + if (modified == 0) {
  85 + modified = 1;
  86 + ftrace_modify_code(addr, ftrace_nop);
  87 + }
  88 + /* ip, module: 0xc0000000, kernel: 0x80000000 */
  89 + new = (ip & 0x40000000) ? lui_v1 : jal_mcount;
  90 +
  91 + return ftrace_modify_code(ip, new);
  92 +}
  93 +
  94 +#define FTRACE_CALL_IP ((unsigned long)(&ftrace_call))
  95 +
  96 +int ftrace_update_ftrace_func(ftrace_func_t func)
  97 +{
  98 + unsigned int new;
  99 +
  100 + new = jump_insn_encode(JAL, (unsigned long)func);
  101 +
  102 + return ftrace_modify_code(FTRACE_CALL_IP, new);
  103 +}
  104 +
  105 +int __init ftrace_dyn_arch_init(void *data)
  106 +{
  107 + /* The return code is retured via data */
  108 + *(unsigned long *)data = 0;
  109 +
  110 + return 0;
  111 +}
  112 +#endif /* CONFIG_DYNAMIC_FTRACE */
arch/mips/kernel/mcount.S
... ... @@ -58,6 +58,33 @@
58 58 move ra, AT
59 59 .endm
60 60  
  61 +#ifdef CONFIG_DYNAMIC_FTRACE
  62 +
  63 +NESTED(ftrace_caller, PT_SIZE, ra)
  64 + .globl _mcount
  65 +_mcount:
  66 + b ftrace_stub
  67 + nop
  68 + lw t0, function_trace_stop
  69 + bnez t0, ftrace_stub
  70 + nop
  71 +
  72 + MCOUNT_SAVE_REGS
  73 +
  74 + move a0, ra /* arg1: next ip, selfaddr */
  75 + .globl ftrace_call
  76 +ftrace_call:
  77 + nop /* a placeholder for the call to a real tracing function */
  78 + move a1, AT /* arg2: the caller's next ip, parent */
  79 +
  80 + MCOUNT_RESTORE_REGS
  81 + .globl ftrace_stub
  82 +ftrace_stub:
  83 + RETURN_BACK
  84 + END(ftrace_caller)
  85 +
  86 +#else /* ! CONFIG_DYNAMIC_FTRACE */
  87 +
61 88 NESTED(_mcount, PT_SIZE, ra)
62 89 lw t0, function_trace_stop
63 90 bnez t0, ftrace_stub
... ... @@ -81,6 +108,8 @@
81 108 ftrace_stub:
82 109 RETURN_BACK
83 110 END(_mcount)
  111 +
  112 +#endif /* ! CONFIG_DYNAMIC_FTRACE */
84 113  
85 114 .set at
86 115 .set reorder
scripts/recordmcount.pl
... ... @@ -295,6 +295,58 @@
295 295 $ld .= " -m elf64_sparc";
296 296 $cc .= " -m64";
297 297 $objcopy .= " -O elf64-sparc";
  298 +} elsif ($arch eq "mips") {
  299 + # To enable module support, we need to enable the -mlong-calls option
  300 + # of gcc for module, after using this option, we can not get the real
  301 + # offset of the calling to _mcount, but the offset of the lui
  302 + # instruction or the addiu one. herein, we record the address of the
  303 + # first one, and then we can replace this instruction by a branch
  304 + # instruction to jump over the profiling function to filter the
  305 + # indicated functions, or swith back to the lui instruction to trace
  306 + # them, which means dynamic tracing.
  307 + #
  308 + # c: 3c030000 lui v1,0x0
  309 + # c: R_MIPS_HI16 _mcount
  310 + # c: R_MIPS_NONE *ABS*
  311 + # c: R_MIPS_NONE *ABS*
  312 + # 10: 64630000 daddiu v1,v1,0
  313 + # 10: R_MIPS_LO16 _mcount
  314 + # 10: R_MIPS_NONE *ABS*
  315 + # 10: R_MIPS_NONE *ABS*
  316 + # 14: 03e0082d move at,ra
  317 + # 18: 0060f809 jalr v1
  318 + #
  319 + # for the kernel:
  320 + #
  321 + # 10: 03e0082d move at,ra
  322 + # 14: 0c000000 jal 0 <loongson_halt>
  323 + # 14: R_MIPS_26 _mcount
  324 + # 14: R_MIPS_NONE *ABS*
  325 + # 14: R_MIPS_NONE *ABS*
  326 + # 18: 00020021 nop
  327 + if ($is_module eq "0") {
  328 + $mcount_regex = "^\\s*([0-9a-fA-F]+):.*\\s_mcount\$";
  329 + } else {
  330 + $mcount_regex = "^\\s*([0-9a-fA-F]+): R_MIPS_HI16\\s+_mcount\$";
  331 + }
  332 + $objdump .= " -Melf-trad".$endian."mips ";
  333 +
  334 + if ($endian eq "big") {
  335 + $endian = " -EB ";
  336 + $ld .= " -melf".$bits."btsmip";
  337 + } else {
  338 + $endian = " -EL ";
  339 + $ld .= " -melf".$bits."ltsmip";
  340 + }
  341 +
  342 + $cc .= " -mno-abicalls -fno-pic -mabi=" . $bits . $endian;
  343 + $ld .= $endian;
  344 +
  345 + if ($bits == 64) {
  346 + $function_regex =
  347 + "^([0-9a-fA-F]+)\\s+<(.|[^\$]L.*?|\$[^L].*?|[^\$][^L].*?)>:";
  348 + $type = ".dword";
  349 + }
298 350 } elsif ($arch eq "microblaze") {
299 351 # Microblaze calls '_mcount' instead of plain 'mcount'.
300 352 $mcount_regex = "^\\s*([0-9a-fA-F]+):.*\\s_mcount\$";