Commit | Line | Data |
---|---|---|
b866c52d SS |
1 | This is a collection of tests for GDB. |
2 | ||
3 | The file gdb/README contains basic instructions on how to run the | |
4 | testsuite, while this file documents additional options and controls | |
5 | that are available. The GDB wiki may also have some pages with ideas | |
6 | and suggestions. | |
7 | ||
8 | ||
9 | Running the Testsuite | |
10 | ********************* | |
11 | ||
12 | There are two ways to run the testsuite and pass additional parameters | |
13 | to DejaGnu. The first is to do `make check' in the main build | |
14 | directory and specifying the makefile variable `RUNTESTFLAGS': | |
15 | ||
87781e84 | 16 | make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp' |
b866c52d SS |
17 | |
18 | The second is to cd to the testsuite directory and invoke the DejaGnu | |
19 | `runtest' command directly. | |
20 | ||
21 | cd testsuite | |
22 | make site.exp | |
87781e84 | 23 | runtest GDB=/usr/bin/gdb |
b866c52d SS |
24 | |
25 | (The `site.exp' file contains a handful of useful variables like host | |
26 | and target triplets, and pathnames.) | |
27 | ||
e352bf0a PA |
28 | Parallel testing |
29 | **************** | |
30 | ||
31 | If not testing with a remote host (in DejaGnu's sense), you can run | |
32 | the GDB test suite in a fully parallel mode. In this mode, each .exp | |
33 | file runs separately and maybe simultaneously. The test suite ensures | |
34 | that all the temporary files created by the test suite do not clash, | |
35 | by putting them into separate directories. This mode is primarily | |
36 | intended for use by the Makefile. | |
37 | ||
38 | For GNU make, the Makefile tries to run the tests in parallel mode if | |
39 | any -j option is given. For a non-GNU make, tests are not | |
40 | parallelized. | |
41 | ||
42 | If RUNTESTFLAGS is not empty, then by default the tests are | |
43 | serialized. This can be overridden by either using the | |
44 | `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL | |
45 | to any non-empty value: | |
46 | ||
acc23c11 PA |
47 | make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver" |
48 | make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1 | |
e352bf0a PA |
49 | |
50 | If you want to use runtest directly instead of using the Makefile, see | |
51 | the description of GDB_PARALLEL below. | |
52 | ||
fb6a751f SDJ |
53 | Racy testcases |
54 | ************** | |
55 | ||
56 | Sometimes, new testcases are added to the testsuite that are not | |
57 | entirely deterministic, and can randomly pass or fail. We call them | |
58 | "racy testcases", and they can be bothersome when one is comparing | |
59 | different testsuite runs. In order to help identifying them, it is | |
60 | possible to run the tests several times in a row and ask the testsuite | |
61 | machinery to analyze the results. To do that, you need to specify the | |
62 | RACY_ITER environment variable to make: | |
63 | ||
64 | make check RACY_ITER=5 -j4 | |
65 | ||
66 | The value assigned to RACY_ITER represents the number of times you | |
67 | wish to run the tests in sequence (in the example above, the entire | |
68 | testsuite will be executed 5 times in a row, in parallel). It is also | |
69 | possible to check just a specific test: | |
70 | ||
71 | make check TESTS='gdb.base/default.exp' RACY_ITER=3 | |
72 | ||
73 | One can also decide to call the Makefile rules by hand inside the | |
74 | gdb/testsuite directory, e.g.: | |
75 | ||
76 | make check-paralell-racy -j4 | |
77 | ||
78 | In which case the value of the DEFAULT_RACY_ITER variable (inside | |
79 | gdb/testsuite/Makefile.in) will be used to determine how many | |
80 | iterations will be run. | |
81 | ||
82 | After running the tests, you shall see a file name 'racy.sum' in the | |
83 | gdb/testsuite directory. You can also inspect the generated *.log and | |
84 | *.sum files by looking into the gdb/testsuite/racy_ouputs directory. | |
85 | ||
86 | If you already have *.sum files generated from previous testsuite runs | |
87 | and you would like to analyze them without having to run the testsuite | |
88 | again, you can also use the 'analyze-racy-logs.py' script directly. | |
89 | It is located in the gdb/testsuite/ directory, and it expects a list | |
90 | of two or more *.sum files to be provided as its argument. For | |
91 | example: | |
92 | ||
93 | ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \ | |
94 | testsuite-02/gdb.sum testsuite-03/gdb.sum | |
95 | ||
96 | The script will output its analysis report to the standard output. | |
97 | ||
b7060614 AH |
98 | Re-running Tests Outside The Testsuite |
99 | ************************************** | |
100 | ||
101 | When running a test, the arguments used to run GDB are saved to gdb.cmd and | |
102 | all commands sent to GDB are saved to gdb.in. As well as being a reference | |
103 | of the commands run, they can be used to manually re-run a test by using | |
104 | the gdb.in file as a batch file to a GDB launched with the arguments in the | |
105 | gdb.cmd file, for example: | |
106 | $(cat outputs/gdb.base/store/gdb.cmd) -x outputs/gdb.base/store/gdb.in | |
107 | ||
108 | Tests that run GDB multiple times will append .1, .2, .3 etc to the end | |
109 | of each .cmd and .in file. | |
110 | ||
111 | When gdbserver is launched as part of a test, a gdbserver.cmd will be created. | |
112 | To re-run these tests, run the contents of gdbserver.cmd in a separate | |
113 | terminal before running gdb, for example: | |
114 | $(cat outputs/gdb.base/store/gdbserver.cmd) | |
115 | Alternatively, if the test is run with GDBSERVER_DEBUG="replay", then this | |
116 | will create a gdbserver.replay file which can be used with the gdbreplay tool, | |
117 | instead of launching gdbserver. | |
118 | ||
71c0c615 YQ |
119 | Running the Performance Tests |
120 | ***************************** | |
121 | ||
122 | GDB Testsuite includes performance test cases, which are not run together | |
123 | with other test cases, because performance test cases are slow and need | |
124 | a quiet system. There are two ways to run the performance test cases. | |
125 | The first is to do `make check-perf' in the main build directory: | |
126 | ||
127 | make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8" | |
128 | ||
129 | The second is to cd to the testsuite directory and invoke the DejaGnu | |
130 | `runtest' command directly. | |
131 | ||
132 | cd testsuite | |
133 | make site.exp | |
134 | runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8 | |
135 | ||
136 | Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They | |
137 | stand for "compile tests only", "run tests only", and "compile and run | |
138 | tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT | |
139 | specify the timeout, which is 3000 in default. The result of | |
140 | performance test is appended in `testsuite/perftest.log'. | |
141 | ||
b866c52d SS |
142 | Testsuite Parameters |
143 | ******************** | |
144 | ||
145 | The following parameters are DejaGNU variables that you can set to | |
146 | affect the testsuite run globally. | |
147 | ||
b866c52d SS |
148 | GDB |
149 | ||
150 | By default, the testsuite exercises the GDB in the build directory, | |
151 | but you can set GDB to be a pathname to a different version. For | |
152 | instance, | |
153 | ||
154 | make check RUNTESTFLAGS=GDB=/usr/bin/gdb | |
155 | ||
156 | runs the testsuite on the GDB in /usr/bin. | |
157 | ||
158 | GDBSERVER | |
159 | ||
160 | You can set GDBSERVER to be a particular GDBserver of interest, so for | |
161 | instance | |
162 | ||
163 | make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver" | |
164 | ||
165 | checks both the installed GDB and GDBserver. | |
166 | ||
167 | INTERNAL_GDBFLAGS | |
168 | ||
169 | Command line options passed to all GDB invocations. | |
170 | ||
171 | The default is "-nw -nx". | |
172 | ||
173 | `-nw' disables any of the windowed interfaces. | |
174 | `-nx' disables ~/.gdbinit, so that it doesn't interfere with | |
175 | the tests. | |
176 | ||
177 | This is actually considered an internal variable, and you | |
178 | won't normally want to change it. However, in some situations, | |
179 | this may be tweaked as a last resort if the testsuite doesn't | |
180 | have direct support for the specifics of your environment. | |
181 | The testsuite does not override a value provided by the user. | |
182 | ||
183 | As an example, when testing an installed GDB that has been | |
184 | configured with `--with-system-gdbinit', like by default, | |
185 | you do not want ~/.gdbinit to interfere with tests, but, you | |
186 | may want the system .gdbinit file loaded. As there's no way to | |
187 | ask the testsuite, or GDB, to load the system gdbinit but | |
188 | not ~/.gdbinit, a workaround is then to remove `-nx' from | |
189 | INTERNAL_GDBFLAGS, and point $HOME at a directory without | |
190 | a .gdbinit. For example: | |
191 | ||
192 | cd testsuite | |
193 | HOME=`pwd` runtest \ | |
194 | GDB=/usr/bin/gdb \ | |
195 | GDBSERVER=/usr/bin/gdbserver \ | |
196 | INTERNAL_GDBFLAGS=-nw | |
197 | ||
198 | GDB_PARALLEL | |
199 | ||
09f2921c | 200 | To use parallel testing mode without using the Makefile, set |
e352bf0a PA |
201 | GDB_PARALLEL on the runtest command line to "yes". Before starting |
202 | the tests, you must ensure that the directories cache, outputs, and | |
203 | temp in the test suite build directory are either empty or have been | |
204 | deleted. cache in particular is used to share data across invocations | |
205 | of runtest, and files there may affect the test results. The Makefile | |
206 | automatically does these deletions. | |
207 | ||
208 | FORCE_PARALLEL | |
209 | ||
210 | Setting FORCE_PARALLEL to any non-empty value forces parallel testing | |
211 | mode even if RUNTESTFLAGS is not empty. | |
b866c52d | 212 | |
51f77c37 PA |
213 | FORCE_SEPARATE_MI_TTY |
214 | ||
215 | Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB | |
216 | in console mode, with MI running on a separate TTY, on a secondary UI | |
217 | started with "new-ui". | |
218 | ||
b866c52d SS |
219 | GDB_INOTIFY |
220 | ||
221 | For debugging parallel mode, it is handy to be able to see when a test | |
222 | case writes to a file outside of its designated output directory. | |
223 | ||
224 | If you have the inotify-tools package installed, you can set the | |
225 | GDB_INOTIFY variable on the runtest command line. This will cause the | |
226 | test suite to watch for parallel-unsafe file creations and report | |
227 | them, both to stdout and in the test suite log file. | |
228 | ||
229 | This setting is only meaningful in conjunction with GDB_PARALLEL. | |
230 | ||
c17ef0d5 DE |
231 | TESTS |
232 | ||
233 | This variable is used to specify which set of tests to run. | |
234 | It is passed to make (not runtest) and its contents are a space separated | |
235 | list of tests to run. | |
236 | ||
237 | If using GNU make then the contents are wildcard-expanded using | |
238 | GNU make's $(wildcard) function. Test paths must be fully specified, | |
239 | relative to the "testsuite" subdirectory. This allows one to run all | |
4992aa20 JM |
240 | tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply |
241 | by using the check-gdb.subdir target in the Makefile. | |
242 | ||
c17ef0d5 DE |
243 | If for some strange reason one wanted to run all tests that begin with |
244 | the letter "d" that is also possible: TESTS="*/d*.exp". | |
245 | ||
246 | Do not write */*.exp to specify all tests (assuming all tests are only | |
247 | nested one level deep, which is not necessarily true). This will pick up | |
248 | .exp files in ancillary directories like "lib" and "config". | |
249 | Instead write gdb.*/*.exp. | |
250 | ||
251 | Example: | |
252 | ||
253 | make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp" | |
254 | ||
255 | If not using GNU make then the value is passed directly to runtest. | |
256 | If not specified, all tests are run. | |
b866c52d | 257 | |
2a31c623 PA |
258 | READ1 |
259 | ||
260 | This make (not runtest) variable is used to specify whether the | |
261 | testsuite preloads the read1.so library into expect. Any non-empty | |
262 | value means true. See "Race detection" below. | |
263 | ||
c7ab0aef SDJ |
264 | GDB_TEST_SOCKETHOST |
265 | ||
266 | This variable can provide the hostname/address that should be used | |
267 | when performing GDBserver-related tests. This is useful in some | |
268 | situations, e.g., when you want to test the IPv6 connectivity of GDB | |
269 | and GDBserver, or when using a different hostname/address is needed. | |
270 | For example, to make GDB and GDBserver use IPv6-only connections, you | |
271 | can do: | |
272 | ||
273 | make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]' | |
274 | ||
275 | Note that only a hostname/address can be provided, without a port | |
276 | number. | |
277 | ||
f63c03b4 SDJ |
278 | TS |
279 | ||
280 | This variable turns on the timestamp printing for each line of "make | |
281 | check". Note that the timestamp will be printed on stdout output | |
282 | only. In other words, there will be no timestamp output on either | |
283 | gdb.sum and gdb.log files. If you would like to enable timestamp | |
284 | printing, you can do: | |
285 | ||
286 | make check TS=1 | |
287 | ||
288 | TS_FORMAT | |
289 | ||
290 | You can provide a custom format for timestamp printing with this | |
291 | variable. The format must be a string compatible with "strftime". | |
292 | This variable is only useful when the TS variable is also provided. | |
293 | If you would like to change the output format of the timestamp, you | |
294 | can do: | |
295 | ||
296 | make check TS=1 TS_FORMAT='[%b %H:%S]' | |
297 | ||
29b52314 AH |
298 | GDB_DEBUG |
299 | ||
300 | When set gdb debug is sent to the file gdb.debug in the test output | |
301 | directory. It should be set to a comma separated list of gdb debug | |
302 | components. | |
303 | For example, to turn on debugging for infrun and target, you can do: | |
304 | ||
305 | make check GDB_DEBUG="infrun,target" | |
306 | ||
dd06d4d6 AH |
307 | GDBSERVER_DEBUG |
308 | ||
b420b89e AH |
309 | When set gdbserver debug is sent to the a file in the test output directory. |
310 | It should be set to a comma separated list of the following options: | |
311 | debug - write gdbserver debug to gdbserver.debug. | |
312 | remote - write gdbserver remote debug to gdbserver.debug. | |
313 | replay - write a replay log to the file gdbserver.replay for use | |
314 | with gdbreplay. | |
315 | Alternatively, it can be set to "all" to turn on all the above | |
316 | For example, to turn on gdbserver debugging, you can do: | |
dd06d4d6 | 317 | |
b420b89e | 318 | make check GDBSERVER_DEBUG="debug,replay" |
dd06d4d6 | 319 | |
2a31c623 PA |
320 | Race detection |
321 | ************** | |
322 | ||
323 | The testsuite includes a mechanism that helps detect test races. | |
324 | ||
325 | For example, say the program running under expect outputs "abcd", and | |
326 | a test does something like this: | |
327 | ||
328 | expect { | |
329 | "a.*c" { | |
330 | } | |
331 | "b" { | |
332 | } | |
333 | "a" { | |
334 | } | |
335 | } | |
336 | ||
337 | Which case happens to match depends on what expect manages to read | |
338 | into its internal buffer in one go. If it manages to read three bytes | |
339 | or more, then the first case matches. If it manages to read two | |
340 | bytes, then the second case matches. If it manages to read only one | |
341 | byte, then the third case matches. | |
342 | ||
343 | To help detect these cases, the race detection mechanism preloads a | |
344 | library into expect that forces the `read' system call to always | |
345 | return at most 1 byte. | |
346 | ||
347 | To enable this, either pass a non-empty value in the READ1 make | |
348 | variable, or use the check-read1 make target instead of check. | |
349 | ||
350 | Examples: | |
351 | ||
352 | make -j10 check-read1 TESTS="*/paginate-*.exp" | |
353 | make -j10 check READ1="1" | |
354 | ||
f8dcc90b TV |
355 | Note: While the intention is to detect races and make otherwise passing tests |
356 | fail, it can also have the effect of making otherwise failing tests pass. | |
357 | This happens f.i. if the test is trying to match a gdb prompt using an end of | |
358 | input marker "${gdb_prompt} $" and there is output after the gdb prompt. This | |
359 | may either pass or fail in normal operation, but using check-read1 will ensure | |
360 | that it passes. | |
361 | ||
b866c52d SS |
362 | Testsuite Configuration |
363 | *********************** | |
364 | ||
365 | It is possible to adjust the behavior of the testsuite by defining | |
366 | the global variables listed below, either in a `site.exp' file, | |
367 | or in a board file. | |
368 | ||
369 | gdb_test_timeout | |
370 | ||
371 | Defining this variable changes the default timeout duration used | |
372 | during communication with GDB. More specifically, the global variable | |
373 | used during testing is `timeout', but this variable gets reset to | |
374 | `gdb_test_timeout' at the beginning of each testcase, which ensures | |
375 | that any local change to `timeout' in a testcase does not affect | |
376 | subsequent testcases. | |
377 | ||
378 | This global variable comes in handy when the debugger is slower than | |
379 | normal due to the testing environment, triggering unexpected `TIMEOUT' | |
380 | test failures. Examples include when testing on a remote machine, or | |
381 | against a system where communications are slow. | |
382 | ||
383 | If not specifically defined, this variable gets automatically defined | |
384 | to the same value as `timeout' during the testsuite initialization. | |
385 | The default value of the timeout is defined in the file | |
386 | `testsuite/config/unix.exp' (at least for Unix hosts; board files may | |
387 | have their own values). | |
388 | ||
8b696e31 YQ |
389 | gdb_reverse_timeout |
390 | ||
391 | Defining this variable changes the default timeout duration when tests | |
392 | under gdb.reverse directory are running. Process record and reverse | |
393 | debugging is so slow that its tests have unexpected `TIMEOUT' test | |
394 | failures. This global variable is useful to bump up the value of | |
395 | `timeout' for gdb.reverse tests and doesn't cause any delay where | |
396 | actual failures happen in the rest of the testsuite. | |
397 | ||
b866c52d SS |
398 | |
399 | Board Settings | |
400 | ************** | |
401 | ||
402 | DejaGNU includes the concept of a "board file", which specifies | |
403 | testing details for a particular target (which are often bare circuit | |
404 | boards, thus the name). | |
405 | ||
406 | In the GDB testsuite specifically, the board file may include a | |
407 | number of "board settings" that test cases may check before deciding | |
408 | whether to exercise a particular feature. For instance, a board | |
409 | lacking any I/O devices, or perhaps simply having its I/O devices | |
410 | not wired up, should set `noinferiorio'. | |
411 | ||
412 | Here are the supported board settings: | |
413 | ||
414 | gdb,cannot_call_functions | |
415 | ||
416 | The board does not support inferior call, that is, invoking inferior | |
417 | functions in GDB. | |
418 | ||
419 | gdb,can_reverse | |
420 | ||
421 | The board supports reverse execution. | |
422 | ||
423 | gdb,no_hardware_watchpoints | |
424 | ||
425 | The board does not support hardware watchpoints. | |
426 | ||
427 | gdb,nofileio | |
428 | ||
429 | GDB is unable to intercept target file operations in remote and | |
430 | perform them on the host. | |
431 | ||
432 | gdb,noinferiorio | |
433 | ||
434 | The board is unable to provide I/O capability to the inferior. | |
435 | ||
436 | gdb,noresults | |
437 | ||
438 | A program will not return an exit code or result code (or the value | |
439 | of the result is undefined, and should not be looked at). | |
440 | ||
441 | gdb,nosignals | |
442 | ||
443 | The board does not support signals. | |
444 | ||
445 | gdb,skip_huge_test | |
446 | ||
447 | Skip time-consuming tests on the board with slow connection. | |
448 | ||
449 | gdb,skip_float_tests | |
450 | ||
451 | Skip tests related to floating point. | |
452 | ||
453 | gdb,use_precord | |
454 | ||
455 | The board supports process record. | |
456 | ||
a25eb028 MR |
457 | gdb_init_command |
458 | gdb_init_commands | |
459 | ||
460 | Commands to send to GDB every time a program is about to be run. The | |
461 | first of these settings defines a single command as a string. The | |
462 | second defines a TCL list of commands being a string each. The commands | |
463 | are sent one by one in a sequence, first from `gdb_init_command', if any, | |
464 | followed by individual commands from `gdb_init_command', if any, in this | |
465 | list's order. | |
466 | ||
b866c52d SS |
467 | gdb_server_prog |
468 | ||
469 | The location of GDBserver. If GDBserver somewhere other than its | |
470 | default location is used in test, specify the location of GDBserver in | |
471 | this variable. The location is a file name for GDBserver, and may be | |
472 | either absolute or relative to the testsuite subdirectory of the build | |
473 | directory. | |
474 | ||
475 | in_proc_agent | |
476 | ||
477 | The location of the in-process agent (used for fast tracepoints and | |
478 | other special tests). If the in-process agent of interest is anywhere | |
479 | other than its default location, set this variable. The location is a | |
480 | filename, and may be either absolute or relative to the testsuite | |
481 | subdirectory of the build directory. | |
482 | ||
483 | noargs | |
484 | ||
485 | GDB does not support argument passing for inferior. | |
486 | ||
487 | no_long_long | |
488 | ||
489 | The board does not support type long long. | |
490 | ||
491 | use_cygmon | |
492 | ||
493 | The board is running the monitor Cygmon. | |
494 | ||
495 | use_gdb_stub | |
496 | ||
497 | The tests are running with a GDB stub. | |
498 | ||
b477a5e6 PA |
499 | exit_is_reliable |
500 | ||
501 | Set to true if GDB can assume that letting the program run to end | |
502 | reliably results in program exits being reported as such, as opposed | |
503 | to, e.g., the program ending in an infinite loop or the board | |
504 | crashing/resetting. If not set, this defaults to $use_gdb_stub. In | |
505 | other words, native targets are assumed reliable by default, and | |
506 | remote stubs assumed unreliable. | |
507 | ||
b866c52d SS |
508 | gdb,predefined_tsv |
509 | ||
510 | The predefined trace state variables the board has. | |
511 | ||
f6512a69 SM |
512 | gdb,no_thread_names |
513 | ||
514 | The target doesn't support thread names. | |
b866c52d | 515 | |
968aa7ae AH |
516 | gdb,pie_flag |
517 | ||
518 | The flag required to force the compiler to produce position-independent | |
519 | executables. | |
520 | ||
521 | gdb,pie_ldflag | |
522 | ||
523 | The flag required to force the linker to produce position-independent | |
524 | executables. | |
525 | ||
6e8b1ab2 JV |
526 | gdb,nopie_flag |
527 | ||
528 | The flag required to force the compiler to produce non-position-independent | |
529 | executables. | |
530 | ||
29b52314 AH |
531 | gdb,debug |
532 | ||
533 | When set gdb debug is sent to the file gdb.debug in the test output | |
534 | directory. It should be set to a comma separated list of gdb debug | |
535 | components. For example, to turn on debugging for infrun and target, set to | |
536 | "infrun,target". | |
537 | ||
dd06d4d6 AH |
538 | gdbserver,debug |
539 | ||
540 | When set gdbserver debug is sent to the file gdbserver.debug in the test | |
b420b89e | 541 | output directory. For valid values see the entry for GDBSERVER_DEBUG. |
dd06d4d6 | 542 | |
b866c52d SS |
543 | Testsuite Organization |
544 | ********************** | |
545 | ||
546 | The testsuite is entirely contained in `gdb/testsuite'. The main | |
547 | directory of the testsuite includes some makefiles and configury, but | |
548 | these are minimal, and used for little besides cleaning up, since the | |
549 | tests themselves handle the compilation of the programs that GDB will | |
550 | run. | |
551 | ||
552 | The file `testsuite/lib/gdb.exp' contains common utility procs useful | |
553 | for all GDB tests, while the directory testsuite/config contains | |
554 | configuration-specific files, typically used for special-purpose | |
555 | definitions of procs like `gdb_load' and `gdb_start'. | |
556 | ||
557 | The tests themselves are to be found in directories named | |
558 | 'testsuite/gdb.* and subdirectories of those. The names of the test | |
559 | files must always end with ".exp". DejaGNU collects the test files by | |
560 | wildcarding in the test directories, so both subdirectories and | |
561 | individual files typically get chosen and run in alphabetical order. | |
562 | ||
563 | The following lists some notable types of subdirectories and what they | |
564 | are for. Since DejaGNU finds test files no matter where they are | |
565 | located, and since each test file sets up its own compilation and | |
566 | execution environment, this organization is simply for convenience and | |
567 | intelligibility. | |
568 | ||
569 | gdb.base | |
570 | ||
571 | This is the base testsuite. The tests in it should apply to all | |
572 | configurations of GDB (but generic native-only tests may live here). | |
573 | The test programs should be in the subset of C that is both valid | |
574 | ANSI/ISO C, and C++. | |
575 | ||
576 | gdb.<lang> | |
577 | ||
578 | Language-specific tests for any language besides C. Examples are | |
9c37b5ae | 579 | gdb.cp for C++ and gdb.rust for Rust. |
b866c52d SS |
580 | |
581 | gdb.<platform> | |
582 | ||
583 | Non-portable tests. The tests are specific to a specific | |
bc23328c | 584 | configuration (host or target), such as eCos. |
b866c52d SS |
585 | |
586 | gdb.arch | |
587 | ||
588 | Architecture-specific tests that are (usually) cross-platform. | |
589 | ||
590 | gdb.<subsystem> | |
591 | ||
592 | Tests that exercise a specific GDB subsystem in more depth. For | |
593 | instance, gdb.disasm exercises various disassemblers, while | |
594 | gdb.stabs tests pathways through the stabs symbol reader. | |
595 | ||
71c0c615 YQ |
596 | gdb.perf |
597 | ||
598 | GDB performance tests. | |
599 | ||
b866c52d SS |
600 | Writing Tests |
601 | ************* | |
602 | ||
603 | In many areas, the GDB tests are already quite comprehensive; you | |
604 | should be able to copy existing tests to handle new cases. Be aware | |
605 | that older tests may use obsolete practices but have not yet been | |
606 | updated. | |
607 | ||
608 | You should try to use `gdb_test' whenever possible, since it includes | |
609 | cases to handle all the unexpected errors that might happen. However, | |
610 | it doesn't cost anything to add new test procedures; for instance, | |
611 | gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test' | |
612 | multiple times. | |
613 | ||
614 | Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even | |
615 | if GDB has several valid responses to a command, you can use | |
616 | `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes | |
617 | internal errors and unexpected prompts. | |
618 | ||
619 | Do not write tests which expect a literal tab character from GDB. On | |
620 | some operating systems (e.g. OpenBSD) the TTY layer expands tabs to | |
621 | spaces, so by the time GDB's output reaches `expect' the tab is gone. | |
622 | ||
623 | The source language programs do *not* need to be in a consistent | |
624 | style. Since GDB is used to debug programs written in many different | |
625 | styles, it's worth having a mix of styles in the testsuite; for | |
626 | instance, some GDB bugs involving the display of source lines might | |
627 | never manifest themselves if the test programs used GNU coding style | |
628 | uniformly. | |
629 | ||
630 | Some testcase results need more detailed explanation: | |
631 | ||
632 | KFAIL | |
633 | ||
634 | Use KFAIL for known problem of GDB itself. You must specify the GDB | |
635 | bug report number, as in these sample tests: | |
636 | ||
637 | kfail "gdb/13392" "continue to marker 2" | |
638 | ||
639 | or | |
640 | ||
641 | setup_kfail gdb/13392 "*-*-*" | |
642 | kfail "continue to marker 2" | |
643 | ||
644 | ||
645 | XFAIL | |
646 | ||
647 | Short for "expected failure", this indicates a known problem with the | |
648 | environment. This could include limitations of the operating system, | |
649 | compiler version, and other components. | |
650 | ||
651 | This example from gdb.base/attach-pie-misread.exp is a sanity check | |
652 | for the target environment: | |
653 | ||
654 | # On x86_64 it is commonly about 4MB. | |
655 | if {$stub_size > 25000000} { | |
656 | xfail "stub size $stub_size is too large" | |
657 | return | |
658 | } | |
659 | ||
660 | You should provide bug report number for the failing component of the | |
661 | environment, if such bug report is available, as with this example | |
662 | referring to a GCC problem: | |
663 | ||
664 | if {[test_compiler_info {gcc-[0-3]-*}] | |
665 | || [test_compiler_info {gcc-4-[0-5]-*}]} { | |
666 | setup_xfail "gcc/46955" *-*-* | |
667 | } | |
668 | gdb_test "python print ttype.template_argument(2)" "&C::c" | |
669 | ||
670 | Note that it is also acceptable, and often preferable, to avoid | |
671 | running the test at all. This is the better option if the limitation | |
672 | is intrinsic to the environment, rather than a bug expected to be | |
673 | fixed in the near future. | |
739b3f1d PA |
674 | |
675 | Local vs Remote vs Native | |
676 | ************************* | |
677 | ||
678 | It's unfortunately easy to get confused in the testsuite about what's | |
679 | native and what's not, what's remote and what's not. The confusion is | |
680 | caused by the overlap in vocabulary between DejaGnu and GDB. | |
681 | ||
682 | From a DejaGnu point of view: | |
683 | ||
684 | - native: the host or target board is considered native if the its | |
685 | triplet is the same as the build system's triplet, | |
686 | ||
687 | - remote: the host or target board is considered remote if it's | |
688 | running on a different machine, and thus require ssh, for example, | |
689 | to run commands, versus simply running commands directly. | |
690 | ||
691 | Note that they are not mutually exclusive, as you can have a remote | |
692 | machine that has the same triplet as the build machine. | |
693 | ||
694 | From a GDB point of view: | |
695 | ||
696 | - native: when GDB uses system calls such as ptrace to interact | |
697 | directly with processes on the same system its running on, | |
698 | ||
699 | - remote: when GDB speaks the RSP (Remote Serial Protocol) with | |
700 | another program doing the ptrace stuff. | |
701 | ||
702 | Note that they are mutually exclusive. An inferior can only be either | |
703 | debugged with the native target, or with the remote target a specific | |
704 | time. | |
705 | ||
706 | That means that there are cases where the target is not remote for | |
707 | DejaGnu, but is remote for GDB (e.g. running GDBserver on the same | |
708 | machine). | |
709 | ||
710 | You can also have a remote target for DejaGnu, but native for GDB | |
711 | (e.g. building on x86 a GDB that runs on ARM and running the | |
712 | testsuite with a remote host). | |
713 | ||
714 | Therefore, care must be taken to check for the right kind of remote. | |
715 | Use [is_remote target] to check whether the DejaGnu target board is | |
716 | remote. When what you really want to know is whether GDB is using the | |
717 | remote protocol, because feature X is only available when GDB debugs | |
718 | natively, check gdb_protocol instead. |