Commit 86151fdf38b3795f292b39defbff39d2684b9c8c
Committed by
Greg Kroah-Hartman
1 parent
e9d376f0fa
Exists in
master
and in
4 other branches
dynamic debug: update docs
updates the documentation for 'dynamic debug' feature. Signed-off-by: Greg Banks <gnb@sgi.com> Signed-off-by: Jason Baron <jbaron@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Showing 2 changed files with 273 additions and 31 deletions Side-by-side Diff
Documentation/dynamic-debug-howto.txt
1 | + | |
2 | +Introduction | |
3 | +============ | |
4 | + | |
5 | +This document describes how to use the dynamic debug (ddebug) feature. | |
6 | + | |
7 | +Dynamic debug is designed to allow you to dynamically enable/disable kernel | |
8 | +code to obtain additional kernel information. Currently, if | |
9 | +CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_debug() calls can be | |
10 | +dynamically enabled per-callsite. | |
11 | + | |
12 | +Dynamic debug has even more useful features: | |
13 | + | |
14 | + * Simple query language allows turning on and off debugging statements by | |
15 | + matching any combination of: | |
16 | + | |
17 | + - source filename | |
18 | + - function name | |
19 | + - line number (including ranges of line numbers) | |
20 | + - module name | |
21 | + - format string | |
22 | + | |
23 | + * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be | |
24 | + read to display the complete list of known debug statements, to help guide you | |
25 | + | |
26 | +Controlling dynamic debug Behaviour | |
27 | +=============================== | |
28 | + | |
29 | +The behaviour of pr_debug()/dev_debug()s are controlled via writing to a | |
30 | +control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs | |
31 | +filesystem, in order to make use of this feature. Subsequently, we refer to the | |
32 | +control file as: <debugfs>/dynamic_debug/control. For example, if you want to | |
33 | +enable printing from source file 'svcsock.c', line 1603 you simply do: | |
34 | + | |
35 | +nullarbor:~ # echo 'file svcsock.c line 1603 +p' > | |
36 | + <debugfs>/dynamic_debug/control | |
37 | + | |
38 | +If you make a mistake with the syntax, the write will fail thus: | |
39 | + | |
40 | +nullarbor:~ # echo 'file svcsock.c wtf 1 +p' > | |
41 | + <debugfs>/dynamic_debug/control | |
42 | +-bash: echo: write error: Invalid argument | |
43 | + | |
44 | +Viewing Dynamic Debug Behaviour | |
45 | +=========================== | |
46 | + | |
47 | +You can view the currently configured behaviour of all the debug statements | |
48 | +via: | |
49 | + | |
50 | +nullarbor:~ # cat <debugfs>/dynamic_debug/control | |
51 | +# filename:lineno [module]function flags format | |
52 | +/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA\040Module\040Removed,\040deregister\040RPC\040RDMA\040transport\012" | |
53 | +/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline\040\040\040\040\040\040\040:\040%d\012" | |
54 | +/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth\040\040\040\040\040\040\040\040\040:\040%d\012" | |
55 | +/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests\040\040\040\040\040:\040%d\012" | |
56 | +... | |
57 | + | |
58 | + | |
59 | +You can also apply standard Unix text manipulation filters to this | |
60 | +data, e.g. | |
61 | + | |
62 | +nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l | |
63 | +62 | |
64 | + | |
65 | +nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l | |
66 | +42 | |
67 | + | |
68 | +Note in particular that the third column shows the enabled behaviour | |
69 | +flags for each debug statement callsite (see below for definitions of the | |
70 | +flags). The default value, no extra behaviour enabled, is "-". So | |
71 | +you can view all the debug statement callsites with any non-default flags: | |
72 | + | |
73 | +nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control | |
74 | +# filename:lineno [module]function flags format | |
75 | +/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process:\040st_sendto\040returned\040%d\012" | |
76 | + | |
77 | + | |
78 | +Command Language Reference | |
79 | +========================== | |
80 | + | |
81 | +At the lexical level, a command comprises a sequence of words separated | |
82 | +by whitespace characters. Note that newlines are treated as word | |
83 | +separators and do *not* end a command or allow multiple commands to | |
84 | +be done together. So these are all equivalent: | |
85 | + | |
86 | +nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' > | |
87 | + <debugfs>/dynamic_debug/control | |
88 | +nullarbor:~ # echo -c ' file svcsock.c line 1603 +p ' > | |
89 | + <debugfs>/dynamic_debug/control | |
90 | +nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' > | |
91 | + <debugfs>/dynamic_debug/control | |
92 | +nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' > | |
93 | + <debugfs>/dynamic_debug/control | |
94 | + | |
95 | +Commands are bounded by a write() system call. If you want to do | |
96 | +multiple commands you need to do a separate "echo" for each, like: | |
97 | + | |
98 | +nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\ | |
99 | +> echo 'file svcsock.c line 1563 +p' > /proc/dprintk | |
100 | + | |
101 | +or even like: | |
102 | + | |
103 | +nullarbor:~ # ( | |
104 | +> echo 'file svcsock.c line 1603 +p' ;\ | |
105 | +> echo 'file svcsock.c line 1563 +p' ;\ | |
106 | +> ) > /proc/dprintk | |
107 | + | |
108 | +At the syntactical level, a command comprises a sequence of match | |
109 | +specifications, followed by a flags change specification. | |
110 | + | |
111 | +command ::= match-spec* flags-spec | |
112 | + | |
113 | +The match-spec's are used to choose a subset of the known dprintk() | |
114 | +callsites to which to apply the flags-spec. Think of them as a query | |
115 | +with implicit ANDs between each pair. Note that an empty list of | |
116 | +match-specs is possible, but is not very useful because it will not | |
117 | +match any debug statement callsites. | |
118 | + | |
119 | +A match specification comprises a keyword, which controls the attribute | |
120 | +of the callsite to be compared, and a value to compare against. Possible | |
121 | +keywords are: | |
122 | + | |
123 | +match-spec ::= 'func' string | | |
124 | + 'file' string | | |
125 | + 'module' string | | |
126 | + 'format' string | | |
127 | + 'line' line-range | |
128 | + | |
129 | +line-range ::= lineno | | |
130 | + '-'lineno | | |
131 | + lineno'-' | | |
132 | + lineno'-'lineno | |
133 | +// Note: line-range cannot contain space, e.g. | |
134 | +// "1-30" is valid range but "1 - 30" is not. | |
135 | + | |
136 | +lineno ::= unsigned-int | |
137 | + | |
138 | +The meanings of each keyword are: | |
139 | + | |
140 | +func | |
141 | + The given string is compared against the function name | |
142 | + of each callsite. Example: | |
143 | + | |
144 | + func svc_tcp_accept | |
145 | + | |
146 | +file | |
147 | + The given string is compared against either the full | |
148 | + pathname or the basename of the source file of each | |
149 | + callsite. Examples: | |
150 | + | |
151 | + file svcsock.c | |
152 | + file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c | |
153 | + | |
154 | +module | |
155 | + The given string is compared against the module name | |
156 | + of each callsite. The module name is the string as | |
157 | + seen in "lsmod", i.e. without the directory or the .ko | |
158 | + suffix and with '-' changed to '_'. Examples: | |
159 | + | |
160 | + module sunrpc | |
161 | + module nfsd | |
162 | + | |
163 | +format | |
164 | + The given string is searched for in the dynamic debug format | |
165 | + string. Note that the string does not need to match the | |
166 | + entire format, only some part. Whitespace and other | |
167 | + special characters can be escaped using C octal character | |
168 | + escape \ooo notation, e.g. the space character is \040. | |
169 | + Examples: | |
170 | + | |
171 | + format svcrdma: // many of the NFS/RDMA server dprintks | |
172 | + format readahead // some dprintks in the readahead cache | |
173 | + format nfsd:\040SETATTR // how to match a format with whitespace | |
174 | + | |
175 | +line | |
176 | + The given line number or range of line numbers is compared | |
177 | + against the line number of each dprintk() callsite. A single | |
178 | + line number matches the callsite line number exactly. A | |
179 | + range of line numbers matches any callsite between the first | |
180 | + and last line number inclusive. An empty first number means | |
181 | + the first line in the file, an empty line number means the | |
182 | + last number in the file. Examples: | |
183 | + | |
184 | + line 1603 // exactly line 1603 | |
185 | + line 1600-1605 // the six lines from line 1600 to line 1605 | |
186 | + line -1605 // the 1605 lines from line 1 to line 1605 | |
187 | + line 1600- // all lines from line 1600 to the end of the file | |
188 | + | |
189 | +The flags specification comprises a change operation followed | |
190 | +by one or more flag characters. The change operation is one | |
191 | +of the characters: | |
192 | + | |
193 | +- | |
194 | + remove the given flags | |
195 | + | |
196 | ++ | |
197 | + add the given flags | |
198 | + | |
199 | += | |
200 | + set the flags to the given flags | |
201 | + | |
202 | +The flags are: | |
203 | + | |
204 | +p | |
205 | + Causes a printk() message to be emitted to dmesg | |
206 | + | |
207 | +Note the regexp ^[-+=][scp]+$ matches a flags specification. | |
208 | +Note also that there is no convenient syntax to remove all | |
209 | +the flags at once, you need to use "-psc". | |
210 | + | |
211 | +Examples | |
212 | +======== | |
213 | + | |
214 | +// enable the message at line 1603 of file svcsock.c | |
215 | +nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' > | |
216 | + <debugfs>/dynamic_debug/control | |
217 | + | |
218 | +// enable all the messages in file svcsock.c | |
219 | +nullarbor:~ # echo -n 'file svcsock.c +p' > | |
220 | + <debugfs>/dynamic_debug/control | |
221 | + | |
222 | +// enable all the messages in the NFS server module | |
223 | +nullarbor:~ # echo -n 'module nfsd +p' > | |
224 | + <debugfs>/dynamic_debug/control | |
225 | + | |
226 | +// enable all 12 messages in the function svc_process() | |
227 | +nullarbor:~ # echo -n 'func svc_process +p' > | |
228 | + <debugfs>/dynamic_debug/control | |
229 | + | |
230 | +// disable all 12 messages in the function svc_process() | |
231 | +nullarbor:~ # echo -n 'func svc_process -p' > | |
232 | + <debugfs>/dynamic_debug/control |
lib/Kconfig.debug
... | ... | @@ -848,59 +848,69 @@ |
848 | 848 | Say N if you are unsure. |
849 | 849 | |
850 | 850 | config DYNAMIC_DEBUG |
851 | - bool "Enable dynamic printk() call support" | |
851 | + bool "Enable dynamic printk() support" | |
852 | 852 | default n |
853 | 853 | depends on PRINTK |
854 | + depends on DEBUG_FS | |
854 | 855 | select PRINTK_DEBUG |
855 | 856 | help |
856 | 857 | |
857 | 858 | Compiles debug level messages into the kernel, which would not |
858 | 859 | otherwise be available at runtime. These messages can then be |
859 | - enabled/disabled on a per module basis. This mechanism implicitly | |
860 | - enables all pr_debug() and dev_dbg() calls. The impact of this | |
861 | - compile option is a larger kernel text size of about 2%. | |
860 | + enabled/disabled based on various levels of scope - per source file, | |
861 | + function, module, format string, and line number. This mechanism | |
862 | + implicitly enables all pr_debug() and dev_dbg() calls. The impact of | |
863 | + this compile option is a larger kernel text size of about 2%. | |
862 | 864 | |
863 | 865 | Usage: |
864 | 866 | |
865 | - Dynamic debugging is controlled by the debugfs file, | |
866 | - dynamic_printk/modules. This file contains a list of the modules that | |
867 | - can be enabled. The format of the file is the module name, followed | |
868 | - by a set of flags that can be enabled. The first flag is always the | |
869 | - 'enabled' flag. For example: | |
867 | + Dynamic debugging is controlled via the 'dynamic_debug/ddebug' file, | |
868 | + which is contained in the 'debugfs' filesystem. Thus, the debugfs | |
869 | + filesystem must first be mounted before making use of this feature. | |
870 | + We refer the control file as: <debugfs>/dynamic_debug/ddebug. This | |
871 | + file contains a list of the debug statements that can be enabled. The | |
872 | + format for each line of the file is: | |
870 | 873 | |
871 | - <module_name> <enabled=0/1> | |
872 | - . | |
873 | - . | |
874 | - . | |
874 | + filename:lineno [module]function flags format | |
875 | 875 | |
876 | - <module_name> : Name of the module in which the debug call resides | |
877 | - <enabled=0/1> : whether the messages are enabled or not | |
876 | + filename : source file of the debug statement | |
877 | + lineno : line number of the debug statement | |
878 | + module : module that contains the debug statement | |
879 | + function : function that contains the debug statement | |
880 | + flags : 'p' means the line is turned 'on' for printing | |
881 | + format : the format used for the debug statement | |
878 | 882 | |
879 | 883 | From a live system: |
880 | 884 | |
881 | - snd_hda_intel enabled=0 | |
882 | - fixup enabled=0 | |
883 | - driver enabled=0 | |
885 | + nullarbor:~ # cat <debugfs>/dynamic_debug/ddebug | |
886 | + # filename:lineno [module]function flags format | |
887 | + fs/aio.c:222 [aio]__put_ioctx - "__put_ioctx:\040freeing\040%p\012" | |
888 | + fs/aio.c:248 [aio]ioctx_alloc - "ENOMEM:\040nr_events\040too\040high\012" | |
889 | + fs/aio.c:1770 [aio]sys_io_cancel - "calling\040cancel\012" | |
884 | 890 | |
885 | - Enable a module: | |
891 | + Example usage: | |
886 | 892 | |
887 | - $echo "set enabled=1 <module_name>" > dynamic_printk/modules | |
893 | + // enable the message at line 1603 of file svcsock.c | |
894 | + nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' > | |
895 | + <debugfs>/dynamic_debug/ddebug | |
888 | 896 | |
889 | - Disable a module: | |
897 | + // enable all the messages in file svcsock.c | |
898 | + nullarbor:~ # echo -n 'file svcsock.c +p' > | |
899 | + <debugfs>/dynamic_debug/ddebug | |
890 | 900 | |
891 | - $echo "set enabled=0 <module_name>" > dynamic_printk/modules | |
901 | + // enable all the messages in the NFS server module | |
902 | + nullarbor:~ # echo -n 'module nfsd +p' > | |
903 | + <debugfs>/dynamic_debug/ddebug | |
892 | 904 | |
893 | - Enable all modules: | |
905 | + // enable all 12 messages in the function svc_process() | |
906 | + nullarbor:~ # echo -n 'func svc_process +p' > | |
907 | + <debugfs>/dynamic_debug/ddebug | |
894 | 908 | |
895 | - $echo "set enabled=1 all" > dynamic_printk/modules | |
909 | + // disable all 12 messages in the function svc_process() | |
910 | + nullarbor:~ # echo -n 'func svc_process -p' > | |
911 | + <debugfs>/dynamic_debug/ddebug | |
896 | 912 | |
897 | - Disable all modules: | |
898 | - | |
899 | - $echo "set enabled=0 all" > dynamic_printk/modules | |
900 | - | |
901 | - Finally, passing "dynamic_printk" at the command line enables | |
902 | - debugging for all modules. This mode can be turned off via the above | |
903 | - disable command. | |
913 | + See Documentation/dynamic-debug-howto.txt for additional information. | |
904 | 914 | |
905 | 915 | source "samples/Kconfig" |
906 | 916 |