Commit | Line | Data |
---|---|---|
86151fdf JB |
1 | |
2 | Introduction | |
3 | ============ | |
4 | ||
5 | This document describes how to use the dynamic debug (ddebug) feature. | |
6 | ||
7 | Dynamic debug is designed to allow you to dynamically enable/disable kernel | |
8 | code to obtain additional kernel information. Currently, if | |
9 | CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_debug() calls can be | |
10 | dynamically enabled per-callsite. | |
11 | ||
12 | Dynamic debug has even more useful features: | |
13 | ||
14 | * Simple query language allows turning on and off debugging statements by | |
15 | matching any combination of: | |
16 | ||
17 | - source filename | |
18 | - function name | |
19 | - line number (including ranges of line numbers) | |
20 | - module name | |
21 | - format string | |
22 | ||
23 | * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be | |
24 | read to display the complete list of known debug statements, to help guide you | |
25 | ||
26 | Controlling dynamic debug Behaviour | |
27 | =============================== | |
28 | ||
29 | The behaviour of pr_debug()/dev_debug()s are controlled via writing to a | |
30 | control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs | |
31 | filesystem, in order to make use of this feature. Subsequently, we refer to the | |
32 | control file as: <debugfs>/dynamic_debug/control. For example, if you want to | |
33 | enable printing from source file 'svcsock.c', line 1603 you simply do: | |
34 | ||
35 | nullarbor:~ # echo 'file svcsock.c line 1603 +p' > | |
36 | <debugfs>/dynamic_debug/control | |
37 | ||
38 | If you make a mistake with the syntax, the write will fail thus: | |
39 | ||
40 | nullarbor:~ # echo 'file svcsock.c wtf 1 +p' > | |
41 | <debugfs>/dynamic_debug/control | |
42 | -bash: echo: write error: Invalid argument | |
43 | ||
44 | Viewing Dynamic Debug Behaviour | |
45 | =========================== | |
46 | ||
47 | You can view the currently configured behaviour of all the debug statements | |
48 | via: | |
49 | ||
50 | nullarbor:~ # cat <debugfs>/dynamic_debug/control | |
51 | # filename:lineno [module]function flags format | |
9898abb3 GB |
52 | /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA Module Removed, deregister RPC RDMA transport\012" |
53 | /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline : %d\012" | |
54 | /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth : %d\012" | |
55 | /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests : %d\012" | |
86151fdf JB |
56 | ... |
57 | ||
58 | ||
59 | You can also apply standard Unix text manipulation filters to this | |
60 | data, e.g. | |
61 | ||
62 | nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l | |
63 | 62 | |
64 | ||
65 | nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l | |
66 | 42 | |
67 | ||
68 | Note in particular that the third column shows the enabled behaviour | |
69 | flags for each debug statement callsite (see below for definitions of the | |
70 | flags). The default value, no extra behaviour enabled, is "-". So | |
71 | you can view all the debug statement callsites with any non-default flags: | |
72 | ||
73 | nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control | |
74 | # filename:lineno [module]function flags format | |
9898abb3 | 75 | /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012" |
86151fdf JB |
76 | |
77 | ||
78 | Command Language Reference | |
79 | ========================== | |
80 | ||
81 | At the lexical level, a command comprises a sequence of words separated | |
82 | by whitespace characters. Note that newlines are treated as word | |
83 | separators and do *not* end a command or allow multiple commands to | |
84 | be done together. So these are all equivalent: | |
85 | ||
86 | nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' > | |
87 | <debugfs>/dynamic_debug/control | |
88 | nullarbor:~ # echo -c ' file svcsock.c line 1603 +p ' > | |
89 | <debugfs>/dynamic_debug/control | |
90 | nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' > | |
91 | <debugfs>/dynamic_debug/control | |
92 | nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' > | |
93 | <debugfs>/dynamic_debug/control | |
94 | ||
95 | Commands are bounded by a write() system call. If you want to do | |
96 | multiple commands you need to do a separate "echo" for each, like: | |
97 | ||
98 | nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\ | |
99 | > echo 'file svcsock.c line 1563 +p' > /proc/dprintk | |
100 | ||
101 | or even like: | |
102 | ||
103 | nullarbor:~ # ( | |
104 | > echo 'file svcsock.c line 1603 +p' ;\ | |
105 | > echo 'file svcsock.c line 1563 +p' ;\ | |
106 | > ) > /proc/dprintk | |
107 | ||
108 | At the syntactical level, a command comprises a sequence of match | |
109 | specifications, followed by a flags change specification. | |
110 | ||
111 | command ::= match-spec* flags-spec | |
112 | ||
113 | The match-spec's are used to choose a subset of the known dprintk() | |
114 | callsites to which to apply the flags-spec. Think of them as a query | |
115 | with implicit ANDs between each pair. Note that an empty list of | |
116 | match-specs is possible, but is not very useful because it will not | |
117 | match any debug statement callsites. | |
118 | ||
119 | A match specification comprises a keyword, which controls the attribute | |
120 | of the callsite to be compared, and a value to compare against. Possible | |
121 | keywords are: | |
122 | ||
123 | match-spec ::= 'func' string | | |
124 | 'file' string | | |
125 | 'module' string | | |
126 | 'format' string | | |
127 | 'line' line-range | |
128 | ||
129 | line-range ::= lineno | | |
130 | '-'lineno | | |
131 | lineno'-' | | |
132 | lineno'-'lineno | |
133 | // Note: line-range cannot contain space, e.g. | |
134 | // "1-30" is valid range but "1 - 30" is not. | |
135 | ||
136 | lineno ::= unsigned-int | |
137 | ||
138 | The meanings of each keyword are: | |
139 | ||
140 | func | |
141 | The given string is compared against the function name | |
142 | of each callsite. Example: | |
143 | ||
144 | func svc_tcp_accept | |
145 | ||
146 | file | |
147 | The given string is compared against either the full | |
148 | pathname or the basename of the source file of each | |
149 | callsite. Examples: | |
150 | ||
151 | file svcsock.c | |
152 | file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c | |
153 | ||
154 | module | |
155 | The given string is compared against the module name | |
156 | of each callsite. The module name is the string as | |
157 | seen in "lsmod", i.e. without the directory or the .ko | |
158 | suffix and with '-' changed to '_'. Examples: | |
159 | ||
160 | module sunrpc | |
161 | module nfsd | |
162 | ||
163 | format | |
164 | The given string is searched for in the dynamic debug format | |
165 | string. Note that the string does not need to match the | |
166 | entire format, only some part. Whitespace and other | |
167 | special characters can be escaped using C octal character | |
168 | escape \ooo notation, e.g. the space character is \040. | |
9898abb3 GB |
169 | Alternatively, the string can be enclosed in double quote |
170 | characters (") or single quote characters ('). | |
86151fdf JB |
171 | Examples: |
172 | ||
173 | format svcrdma: // many of the NFS/RDMA server dprintks | |
174 | format readahead // some dprintks in the readahead cache | |
9898abb3 GB |
175 | format nfsd:\040SETATTR // one way to match a format with whitespace |
176 | format "nfsd: SETATTR" // a neater way to match a format with whitespace | |
177 | format 'nfsd: SETATTR' // yet another way to match a format with whitespace | |
86151fdf JB |
178 | |
179 | line | |
180 | The given line number or range of line numbers is compared | |
181 | against the line number of each dprintk() callsite. A single | |
182 | line number matches the callsite line number exactly. A | |
183 | range of line numbers matches any callsite between the first | |
184 | and last line number inclusive. An empty first number means | |
185 | the first line in the file, an empty line number means the | |
186 | last number in the file. Examples: | |
187 | ||
188 | line 1603 // exactly line 1603 | |
189 | line 1600-1605 // the six lines from line 1600 to line 1605 | |
190 | line -1605 // the 1605 lines from line 1 to line 1605 | |
191 | line 1600- // all lines from line 1600 to the end of the file | |
192 | ||
193 | The flags specification comprises a change operation followed | |
194 | by one or more flag characters. The change operation is one | |
195 | of the characters: | |
196 | ||
197 | - | |
198 | remove the given flags | |
199 | ||
200 | + | |
201 | add the given flags | |
202 | ||
203 | = | |
204 | set the flags to the given flags | |
205 | ||
206 | The flags are: | |
207 | ||
208 | p | |
209 | Causes a printk() message to be emitted to dmesg | |
210 | ||
211 | Note the regexp ^[-+=][scp]+$ matches a flags specification. | |
212 | Note also that there is no convenient syntax to remove all | |
213 | the flags at once, you need to use "-psc". | |
214 | ||
215 | Examples | |
216 | ======== | |
217 | ||
218 | // enable the message at line 1603 of file svcsock.c | |
219 | nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' > | |
220 | <debugfs>/dynamic_debug/control | |
221 | ||
222 | // enable all the messages in file svcsock.c | |
223 | nullarbor:~ # echo -n 'file svcsock.c +p' > | |
224 | <debugfs>/dynamic_debug/control | |
225 | ||
226 | // enable all the messages in the NFS server module | |
227 | nullarbor:~ # echo -n 'module nfsd +p' > | |
228 | <debugfs>/dynamic_debug/control | |
229 | ||
230 | // enable all 12 messages in the function svc_process() | |
231 | nullarbor:~ # echo -n 'func svc_process +p' > | |
232 | <debugfs>/dynamic_debug/control | |
233 | ||
234 | // disable all 12 messages in the function svc_process() | |
235 | nullarbor:~ # echo -n 'func svc_process -p' > | |
236 | <debugfs>/dynamic_debug/control | |
9898abb3 GB |
237 | |
238 | // enable messages for NFS calls READ, READLINK, READDIR and READDIR+. | |
239 | nullarbor:~ # echo -n 'format "nfsd: READ" +p' > | |
240 | <debugfs>/dynamic_debug/control |