Maciej W. Rozycki [Thu, 23 Jul 2020 19:11:29 +0000 (20:11 +0100)]
PR ld/26288: Allow the use of `--just-symbols' with ET_EXEC input
Fix a regression from commit
a87e1817a435 ("Have the linker fail if any
attempt to link in an executable is made.") and do not reject ET_EXEC
input supplied with the `--just-symbols' option. Such use is legitimate
as the file requested is not actually linked and only the symbols are
extracted. Furthermore it is often the most useful application, as
already observed in our documentation for the option, where it allows
"to refer symbolically to absolute locations of memory defined in other
programs."
Provide a set of tests for the use of ET_EXEC with `--just-symbols'.
These are excluded however for SH/PE targets because they complain if a
section's VMA is 0:
ld: zero vma section reloc detected: `.text' #0 f=32795
ld: zero vma section reloc detected: `.data' #1 f=291
and for x86_64/PE targets because they seem to hardwire the VMA:
100000000 12000000 01000000 00000000 00000000 ................
ld/
PR ld/26288
* ldelf.c (ldelf_after_open): Do not reject ET_EXEC input
supplied with `--just-symbols'.
* testsuite/ld-misc/just-symbols.exp: New test script.
* testsuite/ld-misc/just-symbols-1.dd: New test dump.
* testsuite/ld-misc/just-symbols.ld: New test linker script.
* testsuite/ld-misc/just-symbols-0.s: New test source.
* testsuite/ld-misc/just-symbols-1.s: New test source.
Maciej W. Rozycki [Thu, 23 Jul 2020 19:11:29 +0000 (20:11 +0100)]
PR ld/26288: Revert obsolete part of PR ld/26047 fix
Revert commit
a3fc941881e4 ("Stop the linker from accepting executable
ELF files as inputs to other links."), which has been made obsolete by
commit
a87e1817a435 ("Have the linker fail if any attempt to link in an
executable is made."). An earlier check triggers added with the latter
commit making the piece of code removed dead.
ld/
PR ld/26288
Revert:
PR 26047
* ldelf.c (ldelf_after_open): Fail if attempting to link one
executable into another.
Pedro Alves [Thu, 23 Jul 2020 15:29:28 +0000 (16:29 +0100)]
Don't touch frame_info objects if frame cache was reinitialized
This fixes yet another bug exposed by ASAN + multi-target.exp
Running an Asan-enabled GDB against gdb.multi/multi-target.exp exposed
yet another latent GDB bug. See here for the full log:
https://sourceware.org/pipermail/gdb-patches/2020-July/170761.html
As Simon described, the problem is:
- We create a new frame_info object in restore_selected_frame (by
calling find_relative_frame)
- The frame is allocated on the frame_cache_obstack
- In frame_unwind_try_unwinder, we try to find an unwinder for that
frame
- While trying unwinders, memory read fails because the remote target
closes, because of "monitor exit"
- That calls reinit_frame_cache (as shown above), which resets
frame_cache_obstack
- When handling the exception in frame_unwind_try_unwinder, we try to
set some things on the frame_info object (like *this_cache, which
in fact tries to write into frame_info::prologue_cache), but the
frame_info object is no more, it went away with the obstack.
Fix this by maintaining a frame cache generation counter. Then in
exception handling code paths, don't touch frame objects if the
generation is not the same as it was on entry.
This commit generalizes the gdb.server/server-kill.exp testcase and
reuses it to test the scenario in question. The new tests fail
without the GDB fix.
gdb/ChangeLog:
* frame-unwind.c (frame_unwind_try_unwinder): On exception, don't
touch THIS_CACHE/THIS_FRAME if the frame cache was cleared
meanwhile.
* frame.c (frame_cache_generation, get_frame_cache_generation):
New.
(reinit_frame_cache): Increment FRAME_CACHE_GENERATION.
(get_prev_frame_if_no_cycle): On exception, don't touch
PREV_FRAME/THIS_FRAME if the frame cache was cleared meanwhile.
* frame.h (get_frame_cache_generation): Declare.
gdb/testsuite/ChangeLog:
* gdb.server/server-kill.exp (prepare): New, factored out from the
top level.
(kill_server): New.
(test_tstatus, test_unwind_nosyms, test_unwind_syms): New.
(top level) : Call test_tstatus, test_unwind_nosyms, test_unwind_syms.
Tom de Vries [Thu, 23 Jul 2020 11:45:46 +0000 (13:45 +0200)]
[gdb/tui] Fix Wmaybe-uninitialized warning in tui-winsource.c
When compiling with CFLAGS/CXXFLAGS="-O0 -g -Wall" and using g++ 11.0.0, we
run into:
...
src/gdb/tui/tui-winsource.c: In function \
'void tui_update_all_breakpoint_info(breakpoint*)':
src/gdb/tui/tui-winsource.c:427:58: warning: '<unknown>' may be used \
uninitialized [-Wmaybe-uninitialized]
427 | for (tui_source_window_base *win : tui_source_windows ())
| ^
In file included from src/gdb/tui/tui-winsource.c:38:
src/gdb/tui/tui-winsource.h:236:30: note: by argument 1 of type \
'const tui_source_windows*' to 'tui_source_window_iterator \
tui_source_windows::begin() const' declared here
236 | tui_source_window_iterator begin () const
| ^~~~~
src/gdb/tui/tui-winsource.c:427:58: note: '<anonymous>' declared here
427 | for (tui_source_window_base *win : tui_source_windows ())
| ^
...
The warning doesn't make sense for an empty struct, PR gcc/96295 has been
filed about that.
For now, work around the warning by defining a default constructor.
Build on x86_64-linux.
gdb/ChangeLog:
2020-07-23 Tom de Vries <tdevries@suse.de>
PR tui/26282
* tui/tui-winsource.h (struct tui_source_windows::tui_source_windows):
New default constructor.
Andrew Burgess [Tue, 21 Jul 2020 10:21:50 +0000 (11:21 +0100)]
gdb/disassembly: Update to handle non-statement addresses
After the introduction of support for non-statement addresses in the
line table, the output for 'disassemble /m' can be broken in some
cases.
With the /m format disassembly GDB associates a set of addresses with
each line, these addresses are then sorted and printed for each line.
When the non-statement support was added GDB was incorrectly told to
ignore non-statement instructions, and not add these to the result
table. This means that these instructions are completely missing from
the output.
This commit removes the code that caused non-statement lines to be
ignored.
A result of this change is that GDB will now potentially include new
line numbers in the 'disassemble /m' output, lines that previously
were only in the line table as non-statement lines will now appear in
the disassembly output. This feels like an improvement though.
gdb/ChangeLog:
* disasm.c (do_mixed_source_and_assembly_deprecated): Don't
exclude non-statement entries.
gdb/testsuite/ChangeLog:
* gdb.dwarf2/dw2-disasm-over-non-stmt.exp: New file.
Sandra Loosemore [Thu, 23 Jul 2020 03:42:20 +0000 (20:42 -0700)]
Fix more bugs in gdb testglue wrapper handling
In commit
24ac169ac5a918cd82b7485935f0c40a094c625e, this patch:
2020-02-21 Shahab Vahedi <shahab@synopsys.com>
* lib/gdb.exp (gdb_wrapper_init): Reset
"gdb_wrapper_initialized" to 0 if "wrapper_file" does
not exist.
attempted to fix problems finding the gdb test wrapper gdb_tg.o in
some tests that cd to some non-default directory by rebuilding also
the test wrapper in that directory. This had the side-effect of
leaving these .o files in various places in the GDB source directory
tree.
Furthermore, while the tests that cd to some non-default directory
cannot run on remote host, the code that was added to probe for the
presence of the wrapper file was also specific to host == build.
This patch reverts the problematic parts of that commit and replaces
it with forcing use of an absolute (rather than relative) pathname to
the .o file for linking when host == build.
While debugging this patch, I also observed that use of the construct
"[info exists gdb_wrapper_file]" was not reliable for detecting when
that variable had been initialized by gdb_wrapper_init. I rewrote
that so that the variable is always initialized and has a value of an
empty string when no wrapper file is needed.
2020-07-22 Sandra Loosemore <sandra@codesourcery.com>
gdb/testsuite/
* lib/gdb.exp (gdb_wrapper_file, gdb_wrapper_flags):
Initialize to empty string at top level.
(gdb_wrapper_init): Revert check for file existence on build.
Build the wrapper in its default place, not a build-specific
location. When host == build, make the pathname absolute.
(gdb_compile): Delete leftover declaration of
gdb_wrapper_initialized. Check gdb_wrapper_file being an empty
string instead of uninitialized.
GDB Administrator [Thu, 23 Jul 2020 00:00:08 +0000 (00:00 +0000)]
Automatic date update in version.in
Kevin Buettner [Thu, 18 Jun 2020 02:25:47 +0000 (19:25 -0700)]
New core file tests with mappings over existing program memory
This test case was inspired by Pedro's demonstration of a problem
with my v2 patches. It can be found here:
https://sourceware.org/pipermail/gdb-patches/2020-May/168826.html
In a nutshell, my earlier patches could not handle the case in
which a read-only mapping created with mmap() was created at
an address used by other file-backed read-only memory in use by
the process.
This problem has been fixed (for Linux, anyway) by the commit "Use
NT_FILE note section for reading core target memory".
When I run this test without any of my recent corefile patches,
I see these failures:
FAIL: gdb.base/corefile2.exp: kernel core: print/x mbuf_ro[0]@4
FAIL: gdb.base/corefile2.exp: kernel core: print/x mbuf_ro[pagesize-4]@4
FAIL: gdb.base/corefile2.exp: kernel core: print/x mbuf_ro[-3]@6
FAIL: gdb.base/corefile2.exp: kernel core: print/x mbuf_rw[pagesize-3]@6
FAIL: gdb.base/corefile2.exp: kernel core: print/x mbuf_ro[pagesize-3]@6
FAIL: gdb.base/corefile2.exp: maint print core-file-backed-mappings
FAIL: gdb.base/corefile2.exp: gcore core: print/x mbuf_ro[-3]@6
The ones involving mbuf_ro will almost certainly fail when run on
non-Linux systems; I've used setup_xfail on those tests to prevent
them from outright FAILing when not run on Linux. For a time, I
had considered skipping these tests altogether when not run on
Linux, but I changed my mind due to this failure...
FAIL: gdb.base/corefile2.exp: print/x mbuf_rw[pagesize-3]@6
I think it *should* pass without my recent corefile patches. The fact
that it doesn't is likely due to a bug in GDB. The following
interaction with GDB demonstrates the problem:
(gdb) print/x mbuf_rw[pagesize-3]@6
$1 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
(gdb) print/x mbuf_rw[pagesize]@3
$2 = {0x6b, 0x6b, 0x6b}
The last three values in display of $1 should be the same as those
shown by $2. Like this...
(gdb) print/x mbuf_rw[pagesize-3]@6
$1 = {0x0, 0x0, 0x0, 0x6b, 0x6b, 0x6b}
(gdb) print/x mbuf_rw[pagesize]@3
$2 = {0x6b, 0x6b, 0x6b}
That latter output was obtained with the use of all of my current
corefile patches. I see no failures on Linux when running this test
with my current set of corefile patches. I tested 3 architectures:
x86_64, s390x, and aarch64.
I also tested on FreeBSD 12.1-RELEASE. I see the following results
both with and without the current set of core file patches:
# of expected passes 26
# of expected failures 8
Of particular interest is that I did *not* see the problematic mbuf_rw
failure noted earlier (both with and without the core file patches).
I still don't have an explanation for why this failure occurred on
Linux. Prior to running the tests, I had hypothesized that I'd see
this failure on FreeBSD too, but testing shows that this is not the
case.
Also of importance is that we see no FAILs with this test on FreeBSD
which indicates that I XFAILed the correct tests.
This version runs the interesting tests twice, once with a kernel
created core file and another time with a gcore created core file.
It also does a very minimal test of the new command "maint print
core-file-backed-mappings".
gdb/testsuite/ChangeLog:
* gdb.base/corefile2.exp: New file.
* gdb.base/coremaker2.exp: New file.
Kevin Buettner [Sat, 4 Jul 2020 05:09:20 +0000 (22:09 -0700)]
Add documentation for "maint print core-file-backed-mappings"
gdb/ChangeLog:
* NEWS (New commands): Mention new command
"maintenance print core-file-backed-mappings".
gdb/doc/ChangeLog:
* gdb.texinfo (Maintenance Commands): Add documentation for
new command "maintenance print core-file-backed-mappings".
Kevin Buettner [Sat, 4 Jul 2020 04:55:51 +0000 (21:55 -0700)]
Add new command "maint print core-file-backed-mappings"
I wrote a read_core_file_mappings method for FreeBSD and then registered
this gdbarch method. I saw some strange behavior while testing it and
wanted a way to make sure that mappings were being correctly loaded
into corelow.c, so I wrote the new command which is the topic of this
commit. I think it might be occasionally useful for debugging strange
corefile behavior.
With regard to FreeBSD, my work isn't ready yet. Unlike Linux,
FreeBSD puts all mappings into its core file note. And, unlike Linux,
it doesn't dump load segments which occupy no space in the file. So
my (perhaps naive) implementation of a FreeBSD read_core_file_mappings
didn't work all that well: I saw more failures in the corefile2.exp
tests than without it. I think it should be possible to make FreeBSD
work as well as Linux, but it will require doing something with all of
the mappings, not just the file based mappings that I was considering.
In the v4 series, Pedro asked the following:
I don't understand what this command provides that "info proc
mappings" doesn't? Can you give an example of when you'd use this
command over "info proc mappings" ?
On Linux, "info proc mappings" and "maint print core-file-backed-mappings"
will produce similar, possibly identical, output. This need not be
the case for other OSes. E.g. on FreeBSD, had I finished the
implementation, the output from these commands would have been very
different. The FreeBSD "info proc mappings" command would show
additional (non-file-backed) mappings in addition to at least one
additional field (memory permissions) for each mapping.
As noted earlier, I was seeing some unexpected behavior while working
on the FreeBSD implementation and wanted to be certain that the
mappings were being correctly loaded by corelow.c. "info proc
mappings" prints the core file mappings, but doesn't tell us anything
about whether they've been loaded by corelow.c This new maintenance
command directly interrogates the data structures and prints the
values found there.
gdb/ChangeLog:
* corelow.c (gdbcmd.h): Include.
(core_target::info_proc_mappings): New method.
(get_current_core_target): New function.
(maintenance_print_core_file_backed_mappings): New function.
(_initialize_corelow): Add core-file-backed-mappings to
"maint print" commands.
Kevin Buettner [Sat, 4 Jul 2020 03:10:22 +0000 (20:10 -0700)]
Adjust coredump-filter.exp to account for NT_FILE note handling
This commit makes adjustments to coredump-filter.exp to account
for the fact that NT_FILE file-backed mappings are now available
when a core file is loaded. Thus, a test which was expected
to PASS when a memory region was determined to be unavailable
(due to no file-backed mappings being available) will now FAIL
due to those mappings being available from having loaded the
NT_FILE note.
I had originally marked the test as XFAIL, but Mihails Strasuns
suggested a much better approach:
1) First test that it still works if file is accessible in the
filesystem.
2) Temporarily move / rename the file and test that disassembly
doesn't work anymore.
That's what this commit implements.
gdb/testsuite/ChangeLog:
* gdb.base/coredump-filter.exp: Add second
non-Private-Shared-Anon-File test.
(test_disasm): Rename binfile for test which is expected
to fail.
Kevin Buettner [Wed, 1 Jul 2020 13:34:50 +0000 (06:34 -0700)]
gcore command: Place all file-backed mappings in NT_FILE note
When making a core file with the GDB's gcore command on Linux,
the same criteria used for determining which mappings should be
dumped were also being used for determining which entries should
be placed in the NT_FILE note. This is wrong; we want to place
all file-backed mappings in this note.
The predicate function, dump_mapping_p, was used to determine whether
or not to dump a mapping from within linux_find_memory_regions_full.
This commit leaves this predicate in place, but adds a new parameter,
should_dump_mapping_p, to linux_find_memory_regions_full. It then
calls should_dump_mapping_p instead of dump_mapping_p. dump_mapping_p
is passed to linux_find_memory_regions_full at one call site; at the
other call site, dump_note_entry_p is passed instead.
gdb/ChangeLog:
* linux-tdep.c (dump_note_entry_p): New function.
(linux_dump_mapping_p_ftype): New typedef.
(linux_find_memory_regions_full): Add new parameter,
should_dump_mapping_p.
(linux_find_memory_regions): Adjust call to
linux_find_memory_regions_full.
(linux_make_mappings_core_file_notes): Use dump_note_entry_p in
call to linux_find_memory_regions_full.
Kevin Buettner [Tue, 16 Jun 2020 18:39:22 +0000 (11:39 -0700)]
Add test for accessing read-only mmapped data in a core file
This test passes when run using a GDB with my corefile patches. When
run against a GDB without my patches, I see the following failures,
the first of which is due to the test added by this commit:
FAIL: gdb.base/corefile.exp: accessing read-only mmapped data in core file (mapping address not found in core file)
FAIL: gdb.base/corefile.exp: accessing anonymous, unwritten-to mmap data
gdb/testsuite/ChangeLog:
* gdb.base/corefile.exp: Add test "accessing read-only mmapped
data in core file".
* gdb.base/coremaker.c (buf2ro): New global.
(mmapdata): Add a read-only mmap mapping.
Kevin Buettner [Fri, 12 Jun 2020 02:20:03 +0000 (19:20 -0700)]
Use NT_FILE note section for reading core target memory
In his reviews of my v1 and v2 corefile related patches, Pedro
identified two cases which weren't handled by those patches.
In https://sourceware.org/pipermail/gdb-patches/2020-May/168826.html,
Pedro showed that debugging a core file in which mmap() is used to
create a new mapping over an existing file-backed mapping will
produce incorrect results. I.e, for his example, GDB would
show:
(gdb) disassemble main
Dump of assembler code for function main:
0x00000000004004e6 <+0>: push %rbp
0x00000000004004e7 <+1>: mov %rsp,%rbp
=> 0x00000000004004ea <+4>: callq 0x4003f0 <abort@plt>
End of assembler dump.
This sort of looks like it might be correct, but is not due to the
fact that mmap(...MAP_FIXED...) was used to create a mapping (of all
zeros) on top of the .text section. So, the correct result should be:
(gdb) disassemble main
Dump of assembler code for function main:
0x00000000004004e6 <+0>: add %al,(%rax)
0x00000000004004e8 <+2>: add %al,(%rax)
=> 0x00000000004004ea <+4>: add %al,(%rax)
0x00000000004004ec <+6>: add %al,(%rax)
0x00000000004004ee <+8>: add %al,(%rax)
End of assembler dump.
The other case that Pedro found involved an attempted examination of a
particular section in the test case from gdb.base/corefile.exp. On
Fedora 27 or 28, the following behavior may be observed:
(gdb) info proc mappings
Mapped address spaces:
Start Addr End Addr Size Offset objfile
...
0x7ffff7839000 0x7ffff7a38000 0x1ff000 0x1b5000 /usr/lib64/libc-2.27.so
...
(gdb) x/4x 0x7ffff7839000
0x7ffff7839000: Cannot access memory at address 0x7ffff7839000
FYI, this section appears to be unrelocated vtable data. See
https://sourceware.org/pipermail/gdb-patches/2020-May/168331.html for
a detailed analysis.
The important thing here is that GDB should be able to access this
address since it should be backed by the shared library. I.e. it
should do this:
(gdb) x/4x 0x7ffff7839000
0x7ffff7839000: 0x0007ddf0 0x00000000 0x0007dba0 0x00000000
Both of these cases are fixed with this commit.
In a nutshell, this commit opens a "binary" target BFD for each of the
files that are mentioned in an NT_FILE / .note.linuxcore.file note
section. It then uses these mappings instead of the file stratum
mappings that GDB has used in the past.
If this note section doesn't exist or is mangled for some reason, then
GDB will use the file stratum as before. Should this happen, then
we can expect both of the above problems to again be present.
See the code comments in the commit for other details.
gdb/ChangeLog:
* corelow.c (solist.h, unordered_map): Include.
(class core_target): Add field m_core_file_mappings and
method build_file_mappings.
(core_target::core_target): Call build_file_mappings.
(core_target::~core_target): Free memory associated with
m_core_file_mappings.
(core_target::build_file_mappings): New method.
(core_target::xfer_partial): Use m_core_file_mappings
for memory transfers.
* linux-tdep.c (linux_read_core_file_mappings): New
function.
(linux_core_info_proc_mappings): Rewrite to use
linux_read_core_file_mappings.
(linux_init_abi): Register linux_read_core_file_mappings.
Kevin Buettner [Fri, 3 Jul 2020 20:32:08 +0000 (13:32 -0700)]
Add new gdbarch method, read_core_file_mappings
The new gdbarch method, read_core_file_mappings, will be used for
reading file-backed mappings from a core file. It'll be used
for two purposes: 1) to construct a table of file-backed mappings
in corelow.c, and 2) for display of core file mappings.
For Linux, I tried a different approach in which knowledge of the note
format was placed directly in corelow.c. This seemed okay at first;
it was only one note format and the note format was fairly simple.
After looking at FreeBSD's note/mapping reading code, I concluded
that it's best to leave architecture specific details for decoding
the note in (architecture specific) tdep files.
With regard to display of core file mappings, I experimented with
placing the mappings display code in corelow.c. It has access to the
file-backed mappings which were read in when the core file was loaded.
And, better, still common code could be used for all architectures.
But, again, the FreeBSD mapping code convinced me that this was not
the best approach since it has even more mapping info than Linux.
Display code which would work well for Linux will leave out mappings
as well as protection info for mappings.
So, for these reasons, I'm introducing a new gdbarch method for
reading core file mappings.
gdb/ChangeLog:
* arch-utils.c (default_read_core_file_mappings): New function.
* arch-utils.c (default_read_core_file_mappings): Declare.
* gdbarch.sh (read_core_file_mappings): New gdbarch method.
* gdbarch.h, gdbarch.c: Regenerate.
Kevin Buettner [Fri, 12 Jun 2020 01:58:49 +0000 (18:58 -0700)]
Update binary_get_section_contents to seek using section's file position
I have a patch for GDB which opens and reads from BFDs using the
"binary" target. However, for it to work, we need to be able to get a
section's contents based from the file position of that section.
At the moment, reading a section's contents will always read from the
start of the file regardless of where that section is located. While
this was fine for the original use of the "binary" target, it won't
work for my use case. This change shouldn't impact any existing
callers due to the fact that the single .data section is initialized
with a filepos of 0.
bfd/ChangeLog:
* binary.c (binary_get_section_contents): Seek using offset
from section's file position.
Kevin Buettner [Thu, 5 Mar 2020 00:42:43 +0000 (17:42 -0700)]
Test ability to access unwritten-to mmap data in core file
gdb/testsuite/ChangeLog:
PR corefiles/25631
* gdb.base/corefile.exp (accessing anonymous, unwritten-to mmap data):
New test.
* gdb.base/coremaker.c (buf3): New global.
(mmapdata): Add mmap call which uses MAP_ANONYMOUS and MAP_PRIVATE
flags.
Kevin Buettner [Thu, 5 Mar 2020 00:42:42 +0000 (17:42 -0700)]
Provide access to non SEC_HAS_CONTENTS core file sections
Consider the following program:
- - - mkmmapcore.c - - -
static char *buf;
int
main (int argc, char **argv)
{
buf = mmap (NULL, 8192, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
abort ();
}
- - - end mkmmapcore.c - - -
Compile it like this:
gcc -g -o mkmmapcore mkmmapcore.c
Now let's run it from GDB. I've already placed a breakpoint on the
line with the abort() call and have run to that breakpoint.
Breakpoint 1, main (argc=1, argv=0x7fffffffd678) at mkmmapcore.c:11
11 abort ();
(gdb) x/x buf
0x7ffff7fcb000: 0x00000000
Note that we can examine the memory allocated via the call to mmap().
Now let's try debugging a core file created by running this program.
Depending on your system, in order to make a core file, you may have to
run the following as root (or using sudo):
echo core > /proc/sys/kernel/core_pattern
It may also be necessary to do:
ulimit -c unlimited
I'm using Fedora 31. YMMV if you're using one of the BSDs or some other
(non-Linux) system.
This is what things look like when we debug the core file:
[kev@f31-1 tmp]$ gdb -q ./mkmmapcore core.304767
Reading symbols from ./mkmmapcore...
[New LWP 304767]
Core was generated by `/tmp/mkmmapcore'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 return ret;
(gdb) x/x buf
0x7ffff7fcb000: Cannot access memory at address 0x7ffff7fcb000
Note that we can no longer access the memory region allocated by mmap().
Back in 2007, a hack for GDB was added to _bfd_elf_make_section_from_phdr()
in bfd/elf.c:
/* Hack for gdb. Segments that have not been modified do
not have their contents written to a core file, on the
assumption that a debugger can find the contents in the
executable. We flag this case by setting the fake
section size to zero. Note that "real" bss sections will
always have their contents dumped to the core file. */
if (bfd_get_format (abfd) == bfd_core)
newsect->size = 0;
You can find the entire patch plus links to other discussion starting
here:
https://sourceware.org/ml/binutils/2007-08/msg00047.html
This hack sets the size of certain BFD sections to 0, which
effectively causes GDB to ignore them. I think it's likely that the
bug described above existed even before this hack was added, but I
have no easy way to test this now.
The output from objdump -h shows the result of this hack:
25 load13
00000000 00007ffff7fcb000 0000000000000000 00013000 2**12
ALLOC
(The first field, after load13, shows the size of 0.)
Once the hack is removed, the output from objdump -h shows the correct
size:
25 load13
00002000 00007ffff7fcb000 0000000000000000 00013000 2**12
ALLOC
(This is a digression, but I think it's good that objdump will now show
the correct size.)
If we remove the hack from bfd/elf.c, but do nothing to GDB, we'll
see the following regression:
FAIL: gdb.base/corefile.exp: print coremaker_ro
The reason for this is that all sections which have the BFD flag
SEC_ALLOC set, but for which SEC_HAS_CONTENTS is not set no longer
have zero size. Some of these sections have data that can (and should)
be read from the executable. (Sections for which SEC_HAS_CONTENTS
is set should be read from the core file; sections which do not have
this flag set need to either be read from the executable or, failing
that, from the core file using whatever BFD decides is the best value
to present to the user - it uses zeros.)
At present, due to the way that the target strata are traversed when
attempting to access memory, the non-SEC_HAS_CONTENTS sections will be
read as zeroes from the process_stratum (which in this case is the
core file stratum) without first checking the file stratum, which is
where the data might actually be found.
What we should be doing is this:
- Attempt to access core file data for SEC_HAS_CONTENTS sections.
- Attempt to access executable file data if the above fails.
- Attempt to access core file data for non SEC_HAS_CONTENTS sections, if
both of the above fail.
This corresponds to the analysis of Daniel Jacobowitz back in 2007
when the hack was added to BFD:
https://sourceware.org/legacy-ml/binutils/2007-08/msg00045.html
The difference, observed by Pedro in his review of my v1 patches, is
that I'm using "the section flags as proxy for the p_filesz/p_memsz
checks."
gdb/ChangeLog:
PR corefiles/25631
* corelow.c (core_target:xfer_partial): Revise
TARGET_OBJECT_MEMORY case to consider non-SEC_HAS_CONTENTS
case after first checking the stratum beneath the core
target.
(has_all_memory): Return true.
* target.c (raw_memory_xfer_partial): Revise comment
regarding use of has_all_memory.
Kevin Buettner [Thu, 5 Mar 2020 00:42:41 +0000 (17:42 -0700)]
section_table_xfer_memory: Replace section name with callback predicate
This patch is motivated by the need to be able to select sections
that section_table_xfer_memory_partial should consider for memory
transfers. I'll use this facility in the next patch in this series.
section_table_xfer_memory_partial() can currently be passed a section
name which may be used to make name-based selections. This is similar
to what I want to do, except that I want to be able to consider
section flags instead of the name.
I'm replacing the section name parameter with a predicate that,
when passed a pointer to a target_section struct, will return
true if that section should be further considered, or false which
indicates that it shouldn't.
I've converted the one existing use where a non-NULL section
name is passed to section_table_xfer_memory_partial(). Instead
of passing the section name, it now looks like this:
auto match_cb = [=] (const struct target_section *s)
{
return (strcmp (section_name, s->the_bfd_section->name) == 0);
};
return section_table_xfer_memory_partial (readbuf, writebuf,
memaddr, len, xfered_len,
table->sections,
table->sections_end,
match_cb);
The other callers all passed NULL; they've been simplified somewhat
in that they no longer need to pass NULL.
gdb/ChangeLog:
* exec.h (section_table_xfer_memory): Revise declaration,
replacing section name parameter with an optional callback
predicate.
* exec.c (section_table_xfer_memory): Likewise.
* bfd-target.c, exec.c, target.c, corelow.c: Adjust all callers
of section_table_xfer_memory.
Kevin Buettner [Wed, 13 May 2020 00:44:19 +0000 (17:44 -0700)]
Adjust corefile.exp test to show regression after bfd hack removal
In his review of my BZ 25631 patch series, Pedro was unable to
reproduce the regression which should occur after patch #1, "Remove
hack for GDB which sets the section size to 0", is applied.
Pedro was using an ld version older than 2.30. Version 2.30
introduced the linker option -z separate-code. Here's what the man
page has to say about it:
Create separate code "PT_LOAD" segment header in the object. This
specifies a memory segment that should contain only instructions
and must be in wholly disjoint pages from any other data.
In ld version 2.31, use of separate-code became the default for
Linux/x86. So, really, 2.31 or later is required in order to see the
regression that occurs in recent Linux distributions when only the
bfd hack removal patch is applied.
For the test case in question, use of the separate-code linker option
means that the global variable "coremaker_ro" ends up in a separate
load segment (though potentially with other read-only data). The
upshot of this is that when only patch #1 is applied, GDB won't be
able to correctly access coremaker_ro. The reason for this is due
to the fact that this section will now have a non-zero size, but
will not have contents from the core file to find this data.
So GDB will ask BFD for the contents and BFD will respond with
zeroes for anything from those sections. GDB should instead be
looking in the executable for this data. Failing that, it can
then ask BFD for a reasonable value. This is what a later patch
in this series does.
When using ld versions earlier than 2.31 (or 2.30 w/ the
-z separate-code option explicitly provided to the linker), there is
the possibility that coremaker_ro ends up being placed near other data
which is recorded in the core file. That means that the correct value
will end up in the core file, simply because it resides on a page that
the kernel chooses to put in the core file. This is why Pedro wasn't
able to reproduce the regression that should occur after fixing the
BFD hack.
This patch places a big chunk of memory, two pages worth on x86, in
front of "coremaker_ro" to attempt to force it onto another page
without requiring use of that new-fangled linker switch.
Speaking of which, I considered changing the test to use
-z separate-code, but this won't work because it didn't
exist prior to version 2.30. The linker would probably complain
of an unrecognized switch. Also, it likely won't be available in
other linkers not based on current binutils. I.e. it probably won't
work in FreeBSD, NetBSD, etc.
To make this more concrete, this is what *should* happen when
attempting to access coremaker_ro when only patch #1 is applied:
Core was generated by `/mesquite2/sourceware-git/f28-coresegs/bld/gdb/testsuite/outputs/gdb.base/coref'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f68205deefb in raise () from /lib64/libc.so.6
(gdb) p coremaker_ro
$1 = 0
Note that this result is wrong; 201 should have been printed instead.
But that's the point of the rest of the patch series.
However, without this commit, or when using an old Linux distro with
a pre-2.31 ld, this is what you might see instead:
Core was generated by `/mesquite2/sourceware-git/f28-coresegs/bld/gdb/testsuite/outputs/gdb.base/coref'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f63dd658efb in raise () from /lib64/libc.so.6
(gdb) p coremaker_ro
$1 = 201
I.e. it prints the right answer, which sort of makes it seem like the
rest of the series isn't required.
Now, back to the patch itself... what should be the size of the memory
chunk placed before coremaker_ro?
It needs to be at least as big as the page size (PAGE_SIZE) from
the kernel. For x86 and several other architectures this value is
4096. I used MAPSIZE which is defined to be 8192 in coremaker.c.
So it's twice as big as what's currently needed for most Linux
architectures. The constant PAGE_SIZE is available from <sys/user.h>,
but this isn't portable either. In the end, it seemed simpler to
just pick a value and hope that it's big enough. (Running a separate
program which finds the page size via sysconf(_SC_PAGESIZE) and then
passes it to the compilation via a -D switch seemed like overkill
for a case which is rendered moot by recent linker versions.)
Further information can be found here:
https://sourceware.org/pipermail/gdb-patches/2020-May/168168.html
https://sourceware.org/pipermail/gdb-patches/2020-May/168170.html
Thanks to H.J. Lu for telling me about the '-z separate-code' linker
switch.
gdb/testsuite/ChangeLog:
* gdb.base/coremaker.c (filler_ro): New global constant.
Kevin Buettner [Thu, 5 Mar 2020 00:42:40 +0000 (17:42 -0700)]
Remove hack for GDB which sets the section size to 0
This commit removes a hack for GDB which was introduced in 2007.
See:
https://sourceware.org/ml/binutils/2007-08/msg00044.html
That hack mostly allowed GDB's handling of core files to continue to
work without any changes to GDB.
The problem with setting the section size to zero is that GDB won't
know how big that section is/was. Often, this doesn't matter because
the data in question are found in the exec file. But it can happen
that the section describes memory that had been allocated, but never
written to. In this instance, the contents of that memory region are
not written to the core file. Also, since the region in question was
dynamically allocated, it won't appear in the exec file. We don't
want these regions to appear as inaccessible to GDB (since they *were*
accessible when the process was live), so it's important that GDB know
the size of the region.
I've made changes to GDB which correctly handles this case. When
attempting to access memory, GDB will first consider core file data
for which both SEC_ALLOC and SEC_HAS_CONTENTS is set. Next, if that
fails, GDB will attempt to find the data in the exec file. Finally,
if that also fails, GDB will attempt to access memory in the sections
which are flagged as SEC_ALLOC, but not SEC_HAS_CONTENTS.
bfd/ChangeLog:
* elf.c (_bfd_elf_make_section_from_phdr): Remove hack for GDB.
Tom Tromey [Wed, 22 Jul 2020 18:28:33 +0000 (12:28 -0600)]
Fix crash in -stack-list-arguments
-stack-list-arguments will crash when stopped in an Ada procedure that
has an argument with a certain name ("_objectO" -- which can only be
generated by the compiler). The bug occurs because lookup_symbol will
fail in this case.
This patch changes -stack-list-arguments to mirror what is done with
arguments elsewhere. (As an aside, I don't understand why this lookup
is even needed, but I assume it is some stabs thing?)
In the longer term I think it would be good to share this code between
MI and the CLI. However, due to the upcoming release, I preferred a
more local fix.
gdb/ChangeLog
2020-07-22 Tom Tromey <tromey@adacore.com>
* mi/mi-cmd-stack.c (list_args_or_locals): Use
lookup_symbol_search_name.
gdb/testsuite/ChangeLog
2020-07-22 Tom Tromey <tromey@adacore.com>
* gdb.ada/mi_prot.exp: New file.
* gdb.ada/mi_prot/pkg.adb: New file.
* gdb.ada/mi_prot/pkg.ads: New file.
* gdb.ada/mi_prot/prot.adb: New file.
Nick Alcock [Tue, 21 Jul 2020 14:38:08 +0000 (15:38 +0100)]
libctf: fixes for systems on which sizeof (void *) > sizeof (long)
Systems like mingw64 have pointers that can only be represented by 'long
long'. Consistently cast integers stored in pointers through uintptr_t
to cater for this.
libctf/
* ctf-create.c (ctf_dtd_insert): Add uintptr_t casts.
(ctf_dtd_delete): Likewise.
(ctf_dtd_lookup): Likewise.
(ctf_rollback): Likewise.
* ctf-hash.c (ctf_hash_lookup_type): Likewise.
* ctf-types.c (ctf_lookup_by_rawhash): Likewise.
Nick Alcock [Mon, 13 Jul 2020 15:05:15 +0000 (16:05 +0100)]
libctf: fix isspace casts
isspace() notoriously takes an int, not a char. Cast uses
appropriately.
libctf/
* ctf-lookup.c (ctf_lookup_by_name): Adjust.
Nick Alcock [Wed, 1 Jul 2020 19:10:17 +0000 (20:10 +0100)]
libctf, binutils: fix big-endian libctf archive opening
The recent commit "libctf, binutils: support CTF archives like objdump"
broke opening of CTF archives on big-endian platforms.
This didn't affect anyone much before now because the linker never
emitted CTF archives because it wasn't detecting ambiguous types
properly: now it does, and this bug becomes obvious.
Fix trivial.
libctf/
* ctf-archive.c (ctf_arc_bufopen): Endian-swap the archive magic
number if needed.
Nick Alcock [Tue, 14 Jul 2020 17:33:11 +0000 (18:33 +0100)]
ld, testsuite: do not run CTF tests at all on non-ELF for now
Right now, the linker is not emitting CTF sections on (at least some)
non-ELF platforms, because work similar to that done for ELF needs to be
done to each platform in turn to emit linker-generated sections whose
contents are programmatically derived. (Or something better needs to be
done.)
So, for now, the CTF tests will fail on non-ELF for lack of a .ctf
section in the output: so skip the CTF tests there temporarily.
(This is not the same as the permanent skip of the diags tests, which is
done because the input for those is assembler that depends on the ELF
syntax of pseudos like .section: this is only a temporary skip, until
the linker grows support for CTF on more targets.)
ld/
* testsuite/ld-ctf/ctf.exp: Skip on non-ELF for now.
Nick Alcock [Thu, 11 Jun 2020 19:19:07 +0000 (20:19 +0100)]
ld: do not produce one empty output .ctf section for every input .ctf
The trick we use to prevent ld doing as it does for almost all other
sections and copying the input CTF section into the output has recently
broken, causing output to be produced with a valid CTF section followed
by massive numbers of CTF sections, one per .ctf in the input (minus
one, for the one that was filled out by ctf_link). Their size is being
forcibly set to zero, but they're still present, wasting space and
looking ridiculous.
This is not right:
ld/ld-new :
section size addr
.interp 28
4194984
[...]
.bss 21840
6788544
.comment 92 0
.ctf 87242 0
.ctf 0 0
.ctf 0 0
[snip 131 more empty sections]
.gnu.build.attributes 7704
6818576
.debug_aranges 6592 0
.debug_info
4488859 0
.debug_abbrev 150099 0
.debug_line 796759 0
.debug_str 237926 0
.debug_loc
2247302 0
.debug_ranges 237920 0
Total
10865285
The fix is to exclude these unwanted input sections from being present
in the output. We tried this before and it broke things, because if you
exclude all the .ctf sections there isn't going to be one in the output
so there is nowhere to put the deduplicated CTF. The solution to that is
really simple: set SEC_EXCLUDE on *all but one* CTF section. We don't
care which one (they're all the same once their size has been zeroed),
so just pick the first we see.
ld/
* ldlang.c (ldlang_open_ctf): Set SEC_EXCLUDE on all but the
first input .ctf section.
Nick Alcock [Thu, 11 Jun 2020 14:44:48 +0000 (15:44 +0100)]
ld, testsuite: only run CTF tests when ld and GCC support CTF
The CTF testsuite runs GCC to generate CTF that it knows matches the
input .c files before doing a run_dump_test over it. So we need a GCC
capable of doing that, and we need to always avoid running those tests
if libctf was disabled because the linker will never be capable of it.
ld/
* configure.ac (enable_libctf): Substitute it.
* Makefile.am (enablings.exp): New.
(EXTRA_DEJAGNU_SITE_CONFIG): Add it.
(DISTCLEANFILES): Likewise.
* Makefile.in: Regenerate.
* configure: Likewise.
* testsuite/lib/ld-lib.exp (compile_one_cc): New.
(check_ctf_available): Likewise.
(skip_ctf_tests): Likewise.
* testsuite/ld-ctf/ctf.exp: Call skip_ctf_tests.
Egeyar Bagcioglu [Fri, 5 Jun 2020 22:28:22 +0000 (23:28 +0100)]
ld: new CTF testsuite
Uses the new cc option to run_dump_test to compile most tests from C
code, ensuring that the types in the C code accurately describe what the
.d file is testing.
(Some tests, mostly those testing malformed CTF, run directly from .s,
or include both .s and .c.)
ld/
* testsuite/ld-ctf/ctf.exp: New file.
* testsuite/ld-ctf/A-2.c: New file.
* testsuite/ld-ctf/A.c: New file.
* testsuite/ld-ctf/B-2.c: New file.
* testsuite/ld-ctf/B.c: New file.
* testsuite/ld-ctf/C-2.c: New file.
* testsuite/ld-ctf/C.c: New file.
* testsuite/ld-ctf/array-char.c: New file.
* testsuite/ld-ctf/array-int.c: New file.
* testsuite/ld-ctf/array.d: New file.
* testsuite/ld-ctf/child-float.c: New file.
* testsuite/ld-ctf/child-int.c: New file.
* testsuite/ld-ctf/conflicting-cycle-1.B-1.d: New file.
* testsuite/ld-ctf/conflicting-cycle-1.B-2.d: New file.
* testsuite/ld-ctf/conflicting-cycle-1.parent.d: New file.
* testsuite/ld-ctf/conflicting-cycle-2.A-1.d: New file.
* testsuite/ld-ctf/conflicting-cycle-2.A-2.d: New file.
* testsuite/ld-ctf/conflicting-cycle-2.parent.d: New file.
* testsuite/ld-ctf/conflicting-cycle-3.C-1.d: New file.
* testsuite/ld-ctf/conflicting-cycle-3.C-2.d: New file.
* testsuite/ld-ctf/conflicting-cycle-3.parent.d: New file.
* testsuite/ld-ctf/conflicting-enums.d: New file.
* testsuite/ld-ctf/conflicting-typedefs.d: New file.
* testsuite/ld-ctf/cross-tu-1.c: New file.
* testsuite/ld-ctf/cross-tu-2.c: New file.
* testsuite/ld-ctf/cross-tu-conflicting-2.c: New file.
* testsuite/ld-ctf/cross-tu-cyclic-1.c: New file.
* testsuite/ld-ctf/cross-tu-cyclic-2.c: New file.
* testsuite/ld-ctf/cross-tu-cyclic-3.c: New file.
* testsuite/ld-ctf/cross-tu-cyclic-4.c: New file.
* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: New file.
* testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d: New file.
* testsuite/ld-ctf/cross-tu-into-cycle.d: New file.
* testsuite/ld-ctf/cross-tu-noncyclic.d: New file.
* testsuite/ld-ctf/cycle-1.c: New file.
* testsuite/ld-ctf/cycle-1.d: New file.
* testsuite/ld-ctf/cycle-2.A.d: New file.
* testsuite/ld-ctf/cycle-2.B.d: New file.
* testsuite/ld-ctf/cycle-2.C.d: New file.
* testsuite/ld-ctf/diag-ctf-version-0.d: New file.
* testsuite/ld-ctf/diag-ctf-version-0.s: New file.
* testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d: New file.
* testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s: New file.
* testsuite/ld-ctf/diag-ctf-version-f.d: New file.
* testsuite/ld-ctf/diag-ctf-version-f.s: New file.
* testsuite/ld-ctf/diag-cttname-invalid.d: New file.
* testsuite/ld-ctf/diag-cttname-invalid.s: New file.
* testsuite/ld-ctf/diag-cttname-null.d: New file.
* testsuite/ld-ctf/diag-cttname-null.s: New file.
* testsuite/ld-ctf/diag-cuname.d: New file.
* testsuite/ld-ctf/diag-cuname.s: New file.
* testsuite/ld-ctf/diag-decompression-failure.d: New file.
* testsuite/ld-ctf/diag-decompression-failure.s: New file.
* testsuite/ld-ctf/diag-parlabel.d: New file.
* testsuite/ld-ctf/diag-parlabel.s: New file.
* testsuite/ld-ctf/diag-parname.d: New file.
* testsuite/ld-ctf/diag-parname.s: New file.
* testsuite/ld-ctf/diag-unsupported-flag.d: New file.
* testsuite/ld-ctf/diag-unsupported-flag.s: New file.
* testsuite/ld-ctf/diag-wrong-magic-number-mixed.d: New file.
* testsuite/ld-ctf/diag-wrong-magic-number.d: New file.
* testsuite/ld-ctf/diag-wrong-magic-number.s: New file.
* testsuite/ld-ctf/enum-2.c: New file.
* testsuite/ld-ctf/enum.c: New file.
* testsuite/ld-ctf/function.c: New file.
* testsuite/ld-ctf/function.d: New file.
* testsuite/ld-ctf/slice.c: New file.
* testsuite/ld-ctf/slice.d: New file.
* testsuite/ld-ctf/super-sub-cycles.c: New file.
* testsuite/ld-ctf/super-sub-cycles.d: New file.
* testsuite/ld-ctf/typedef-int.c: New file.
* testsuite/ld-ctf/typedef-long.c: New file.
* testsuite/ld-ctf/union-1.c: New file.
Nick Alcock [Thu, 11 Jun 2020 14:48:55 +0000 (15:48 +0100)]
binutils, testsuite: allow compilation before doing run_dump_test
The CTF assembler emitted by GCC has architecture-dependent pseudos in
it, and is (obviously) tightly tied to a particular set of C source
files with specific types in them. The CTF tests do run_dump_test on
some candidate input, link it using the run_dump_test ld machinery, and
compare objdump --ctf output. To avoid skew, we'd like to be able
to easily regenerate the .s being scanned so that the .c doesn't get
out of sync with it, but since GCC emits arch-dependent pseudos, we
are forced to hand-hack the output every time (quite severely on some
arches, like x86-32 and -64, where every single pseudo used is not only
arch-dependent but undocumented).
To avoid this, teach run_dump_test how to optionally compile things
given new, optional additional flags passed in in the cc option.
Only sources with the .c suffix are compiled, so there is no effect on
any existing tests. The .s files go into the tmpdir, from which
existing run_dump_test code picks them up as usual.
binutils/
* testsuite/lib/binutils-common.exp (run_dump_test): Add 'cc'
option.
Nick Alcock [Fri, 5 Jun 2020 22:18:06 +0000 (23:18 +0100)]
ld: new options --ctf-variables and --ctf-share-types
libctf recently changed to make it possible to not emit the CTF
variables section. Make this the default for ld: the variables section
is a simple name -> type mapping, and the names can be quite voluminous.
Nothing in the variables section appears in the symbol table, by
definition, so GDB cannot make use of them: special-purpose projects
that implement their own analogues of symbol table lookup can do so, but
they'll need to tell the linker to emit the variables section after all.
The new --ctf-variables option does this.
The --ctf-share-types option (valid values "share-duplicated" and
"share-unconflicted") allow the caller to specify the CTF link mode.
Most users will want share-duplicated, since it allows for more
convenient debugging: but very large projects composed of many decoupled
components may want to use share-unconflicted mode, which places types
that appear in only one TU into per-TU dicts. (They may also want to
relink the CTF using the ctf_link API and cu-mapping, to make their
"components" larger than a single TU. Right now the linker does not
expose the CU-mapping machinery. Perhaps it should in future to make
this use case easier.)
For now, giving the linker the ability to emit share-duplicated CTF lets
us add testcases for that mode to the testsuite.
ld/
* ldlex.h (option_values) <OPTION_CTF_VARIABLES,
OPTION_NO_CTF_VARIABLES, OPTION_CTF_SHARE_TYPES>: New.
* ld.h (ld_config_type) <ctf_variables, ctf_share_duplicated>:
New fields.
* ldlang.c (lang_merge_ctf): Use them.
* lexsup.c (ld_options): Add ctf-variables, no-ctf-variables,
ctf-share-types.
(parse_args) <OPTION_CTF_VARIABLES, OPTION_NO_CTF_VARIABLES,
OPTION_CTF_SHARE_TYPES>: New cases.
* ld.texi: Document new options.
* NEWS: Likewise.
Egeyar Bagcioglu [Wed, 15 Apr 2020 13:37:34 +0000 (15:37 +0200)]
ld: Reformat CTF errors into warnings.
ld/
* ldlang.c (lang_merge_ctf): Turn errors into warnings.
Fix a comment typo.
(lang_write_ctf): Turn an error into a warning.
(ldlang_open_ctf): Reformat warnings. Fix printing file names.
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
Nick Alcock [Fri, 5 Jun 2020 22:13:01 +0000 (23:13 +0100)]
binutils: objdump: ctf: drop incorrect linefeeds
The CTF objdumping code is adding linefeeds in calls to non_fatal, which
is wrong and looks ugly.
binutils/
* objdump.c (dump_ctf_archive_member): Remove linefeeds.
(dump_ctf): Likewise.
Nick Alcock [Fri, 5 Jun 2020 21:57:06 +0000 (22:57 +0100)]
libctf, link: tie in the deduplicating linker
This fairly intricate commit connects up the CTF linker machinery (which
operates in terms of ctf_archive_t's on ctf_link_inputs ->
ctf_link_outputs) to the deduplicator (which operates in terms of arrays
of ctf_file_t's, all the archives exploded).
The nondeduplicating linker is retained, but is not called unless the
CTF_LINK_NONDEDUP flag is passed in (which ld never does), or the
environment variable LD_NO_CTF_DEDUP is set. Eventually, once we have
confidence in the much-more-complex deduplicating linker, I hope the
nondeduplicating linker can be removed.
In brief, what this does is traverses each input archive in
ctf_link_inputs, opening every member (if not already open) and tying
child dicts to their parents, shoving them into an array and
constructing a corresponding parents array that tells the deduplicator
which dict is the parent of which child. We then call ctf_dedup and
ctf_dedup_emit with that array of inputs, taking the outputs that result
and putting them into ctf_link_outputs where the rest of the CTF linker
expects to find them, then linking in the variables just as is done by
the nondeduplicating linker.
It also implements much of the CU-mapping side of things. The problem
CU-mapping introduces is that if you map many input CUs into one output,
this is saying that you want many translation units to produce at most
one child dict if conflicting types are found in any of them. This
means you can suddenly have multiple distinct types with the same name
in the same dict, which libctf cannot really represent because it's not
something you can do with C translation units.
The deduplicator machinery already committed does as best it can with
these, hiding types with conflicting names rather than making child
dicts out of them: but we still need to call it. This is done similarly
to the main link, taking the inputs (one CU output at a time),
deduplicating them, taking the output and making it an input to the
final link. Two (significant) optimizations are done: we share atoms
tables between all these links and the final link (so e.g. all type hash
values are shared, all decorated type names, etc); and any CU-mapped
links with only one input (and no child dicts) doesn't need to do
anything other than renaming the CU: the CU-mapped link phase can be
skipped for it. Put together, large CU-mapped links can save 50% of
their memory usage and about as much time (and the memory usage for
CU-mapped links is significant, because all those output CUs have to
have all their types stored in memory all at once).
include/
* ctf-api.h (CTF_LINK_NONDEDUP): New, turn off the
deduplicator.
libctf/
* ctf-impl.h (ctf_list_splice): New.
* ctf-util.h (ctf_list_splice): Likewise.
* ctf-link.c (link_sort_inputs_cb_arg_t): Likewise.
(ctf_link_sort_inputs): Likewise.
(ctf_link_deduplicating_count_inputs): Likewise.
(ctf_link_deduplicating_open_inputs): Likewise.
(ctf_link_deduplicating_close_inputs): Likewise.
(ctf_link_deduplicating_variables): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
(ctf_link_deduplicating): Likewise.
(ctf_link): Call it.
Nick Alcock [Fri, 5 Jun 2020 21:52:41 +0000 (22:52 +0100)]
libctf, link: add CTF_LINK_OMIT_VARIABLES_SECTION
This flag (not used anywhere yet) causes the variables section to be
omitted from the output CTF dict.
include/
* ctf-api.h (CTF_LINK_OMIT_VARIABLES_SECTION): New.
libctf/
* ctf-link.c (ctf_link_one_input_archive_member): Check
CTF_LINK_OMIT_VARIABLES_SECTION.
Nick Alcock [Fri, 5 Jun 2020 17:35:46 +0000 (18:35 +0100)]
libctf, dedup: add deduplicator
This adds the core deduplicator that the ctf_link machinery calls
(possibly repeatedly) to link the CTF sections: it takes an array
of input ctf_file_t's and another array that indicates which entries in
the input array are parents of which other entries, and returns an array
of outputs. The first output is always the ctf_file_t on which
ctf_link/ctf_dedup/etc was called: the other outputs are child dicts
that have the first output as their parent.
include/
* ctf-api.h (CTF_LINK_SHARE_DUPLICATED): No longer unimplemented.
libctf/
* ctf-impl.h (ctf_type_id_key): New, the key in the
cd_id_to_file_t.
(ctf_dedup): New, core deduplicator state.
(ctf_file_t) <ctf_dedup>: New.
<ctf_dedup_atoms>: New.
<ctf_dedup_atoms_alloc>: New.
(ctf_hash_type_id_key): New prototype.
(ctf_hash_eq_type_id_key): Likewise.
(ctf_dedup_atoms_init): Likewise.
* ctf-hash.c (ctf_hash_eq_type_id_key): New.
(ctf_dedup_atoms_init): Likewise.
* ctf-create.c (ctf_serialize): Adjusted.
(ctf_add_encoded): No longer static.
(ctf_add_reftype): Likewise.
* ctf-open.c (ctf_file_close): Destroy the
ctf_dedup_atoms_alloc.
* ctf-dedup.c: New file.
* ctf-decls.h [!HAVE_DECL_STPCPY]: Add prototype.
* configure.ac: Check for stpcpy.
* Makefile.am: Add it.
* Makefile.in: Regenerate.
* config.h.in: Regenerate.
* configure: Regenerate.
Nick Alcock [Fri, 5 Jun 2020 17:38:03 +0000 (18:38 +0100)]
libctf, dedup: add new configure option --enable-libctf-hash-debugging
Add a new debugging configure option, --enable-libctf-hash-debugging,
off by default, which lets you configure in expensive internal
consistency checks and enable the printing of debugging output when
LIBCTF_DEBUG=t before type deduplication has happened.
In this commit we just add the option and cause it to turn ctf_assert
into a real, hard assert for easier debugging.
libctf/
* configure.ac: Add --enable-libctf-hash-debugging.
* aclocal.m4: Pull in enable.m4, for GCC_ENABLE.
* Makefile.in: Regenerated.
* configure: Likewise.
* config.h.in: Likewise.
* ctf-impl.h [ENABLE_LIBCTF_HASH_DEBUGGING]
(ctf_assert): Define to assert.
Nick Alcock [Fri, 5 Jun 2020 20:10:37 +0000 (21:10 +0100)]
libctf: add SHA-1 support for libctf
This very thin abstraction layer provides SHA-1ing facilities to all of
libctf, almost all inlined wrappers around the libiberty functionality
other than ctf_sha1_fini.
The deduplicator will use this to recursively hash types to prove their
identity.
libctf/
* ctf-sha1.h: New, inline wrappers around sha1_init_ctx and
sha1_process_bytes.
* ctf-impl.h: Include it.
(ctf_sha1_init): New.
(ctf_sha1_add): Likewise.
(ctf_sha1_fini): Likewise.
* ctf-sha1.c: New, non-inline wrapper around sha1_finish_ctx
producing strings.
* Makefile.am: Add file.
* Makefile.in: Regenerate.
Nick Alcock [Fri, 5 Jun 2020 17:15:26 +0000 (18:15 +0100)]
libctf, link: add the ability to filter out variables from the link
The CTF variables section (containing variables that have no
corresponding symtab entries) can cause the string table to get very
voluminous if the names of variables are long. Some callers want to
filter out particular variables they know they won't need.
So add a "variable filter" callback that does that: it's passed the name
of the variable and a corresponding ctf_file_t / ctf_id_t pair, and
should return 1 to filter it out.
ld doesn't use this machinery yet, but we could easily add it later if
desired. (But see later for a commit that turns off CTF variable-
section linking in ld entirely by default.)
include/
* ctf-api.h (ctf_link_variable_filter_t): New.
(ctf_link_set_variable_filter): Likewise.
libctf/
* libctf.ver (ctf_link_set_variable_filter): Add.
* ctf-impl.h (ctf_file_t) <ctf_link_variable_filter>: New.
<ctf_link_variable_filter_arg>: Likewise.
* ctf-create.c (ctf_serialize): Adjust.
* ctf-link.c (ctf_link_set_variable_filter): New, set it.
(ctf_link_one_variable): Call it if set.
Nick Alcock [Wed, 11 Mar 2020 00:01:41 +0000 (00:01 +0000)]
libctf, link: fix spurious conflicts of variables in the variable section
When we link a CTF variable, we check to see if it already exists in the
parent dict first: if it does, and it has a type the same as the type we
would populate it with, we assume we don't need to do anything:
otherwise, we populate it in a per-CU child.
Or that's what we should be doing. Instead, we check if the type is the
same as the type in *source dict*, which is going to be a completely
different value! So we end up concluding all variables are conflicting,
bloating up output possibly quite a lot (variables aren't big in and of
themselves, but each drags around a strtab entry, and CTF dicts in a CTF
archive do not share their strtabs -- one of many problems with CTF
archives as presently constituted.)
Fix trivial: check the right type.
libctf/
* ctf-link.c (ctf_link_one_variable): Check the dst_type for
conflicts, not the source type.
Nick Alcock [Fri, 5 Jun 2020 16:36:16 +0000 (17:36 +0100)]
libctf, link: redo cu-mapping handling
Now a bunch of stuff that doesn't apply to ld or any normal use of
libctf, piled into one commit so that it's easier to ignore.
The cu-mapping machinery associates incoming compilation unit names with
outgoing names of CTF dictionaries that should correspond to them, for
non-gdb CTF consumers that would like to group multiple TUs into a
single child dict if conflicting types are found in it (the existing use
case is one kernel module, one child CTF dict, even if the kernel module
is composed of multiple CUs).
The upcoming deduplicator needs to track not only the mapping from
incoming CU name to outgoing dict name, but the inverse mapping from
outgoing dict name to incoming CU name, so it can work over every CTF
dict we might see in the output and link into it.
So rejig the ctf-link machinery to do that. Simultaneously (because
they are closely associated and were written at the same time), we add a
new CTF_LINK_EMPTY_CU_MAPPINGS flag to ctf_link, which tells the
ctf_link machinery to create empty child dicts for each outgoing CU
mapping even if no CUs that correspond to it exist in the link. This is
a bit (OK, quite a lot) of a waste of space, but some existing consumers
require it. (Nobody else should use it.)
Its value is not consecutive with existing CTF_LINK flag values because
we're about to add more flags that are conceptually closer to the
existing ones than this one is.
include/
* ctf-api.h (CTF_LINK_EMPTY_CU_MAPPINGS): New.
libctf/
* ctf-impl.h (ctf_file_t): Improve comments.
<ctf_link_cu_mapping>: Split into...
<ctf_link_in_cu_mapping>: ... this...
<ctf_link_out_cu_mapping>: ... and this.
* ctf-create.c (ctf_serialize): Adjust.
* ctf-open.c (ctf_file_close): Likewise.
* ctf-link.c (ctf_create_per_cu): Look things up in the
in_cu_mapping instead of the cu_mapping.
(ctf_link_add_cu_mapping): The deduplicating link will define
what happens if many FROMs share a TO.
(ctf_link_add_cu_mapping): Create in_cu_mapping and
out_cu_mapping. Do not create ctf_link_outputs here any more, or
create per-CU dicts here: they are already created when needed.
(ctf_link_one_variable): Log a debug message if we skip a
variable due to its type being concealed in a CU-mapped link.
(This is probably too common a case to make into a warning.)
(ctf_link): Create empty per-CU dicts if requested.
Nick Alcock [Thu, 4 Jun 2020 18:49:36 +0000 (19:49 +0100)]
libctf, link: fix ctf_link_write fd leak
We were leaking the fd on every invocation.
libctf/
* ctf-link.c (ctf_link_write): Close the fd.
Nick Alcock [Thu, 4 Jun 2020 18:28:52 +0000 (19:28 +0100)]
libctf, link: add lazy linking: clean up input members: err/warn cleanup
This rather large and intertwined pile of changes does three things:
First, it transitions from dprintf to ctf_err_warn for things the user might
care about: this one file is the major impetus for the ctf_err_warn
infrastructure, because things like file names are crucial in linker
error messages, and errno values are utterly incapable of
communicating them
Second, it stabilizes the ctf_link APIs: you can now call
ctf_link_add_ctf without a CTF argument (only a NAME), to lazily
ctf_open the file with the given NAME when needed, and close it as soon
as possible, to save memory. This is not an API change because a null
CTF argument was prohibited before now.
Since getting CTF directly from files uses ctf_open, passing in only a
NAME requires use of libctf, not libctf-nobfd. The linker's behaviour
is unchanged, as it still passes in a ctf_archive_t as before.
This also let us fix a leak: we were opening ctf_archives and their
containing ctf_files, then only closing the files and leaving the
archives open.
Third, this commit restructures the ctf_link_in_member argument used by
the CTF linking machinery and adjusts its users accordingly.
We drop two members:
- arcname, which is difficult to construct and then only used in error
messages (that were only dprintf()ed, so never seen!)
- share_mode, since we store the flags passed to ctf_link (including the
share mode) in a new ctf_file_t.ctf_link_flags to help dedup get hold
of it
We rename others whose existing names were fairly dreadful:
- done_main_member -> done_parent, using consistent terminology for .ctf
as the parent of all archive members
- main_input_fp -> in_fp_parent, likewise
- file_name -> in_file_name, likewise
We add one new member, cu_mapped.
Finally, we move the various frees of things like mapping table data to
the top-level ctf_link, since deduplicating links will want to do that
too.
include/
* ctf-api.h (ECTF_NEEDSBFD): New.
(ECTF_NERR): Adjust.
(ctf_link): Rename share_mode arg to flags.
libctf/
* Makefile.am: Set -DNOBFD=1 in libctf-nobfd, and =0 elsewhere.
* Makefile.in: Regenerated.
* ctf-impl.h (ctf_link_input_name): New.
(ctf_file_t) <ctf_link_flags>: New.
* ctf-create.c (ctf_serialize): Adjust accordingly.
* ctf-link.c: Define ctf_open as weak when PIC.
(ctf_arc_close_thunk): Remove unnecessary thunk.
(ctf_file_close_thunk): Likewise.
(ctf_link_input_name): New.
(ctf_link_input_t): New value of the ctf_file_t.ctf_link_input.
(ctf_link_input_close): Adjust accordingly.
(ctf_link_add_ctf_internal): New, split from...
(ctf_link_add_ctf): ... here. Return error if lazy loading of
CTF is not possible. Change to just call...
(ctf_link_add): ... this new function.
(ctf_link_add_cu_mapping): Transition to ctf_err_warn. Drop the
ctf_file_close_thunk.
(ctf_link_in_member_cb_arg_t) <file_name> Rename to...
<in_file_name>: ... this.
<arcname>: Drop.
<share_mode>: Likewise (migrated to ctf_link_flags).
<done_main_member>: Rename to...
<done_parent>: ... this.
<main_input_fp>: Rename to...
<in_fp_parent>: ... this.
<cu_mapped>: New.
(ctf_link_one_type): Adjuwt accordingly. Transition to
ctf_err_warn, removing a TODO.
(ctf_link_one_variable): Note a case too common to warn about.
Report in the debug stream if a cu-mapped link prevents addition
of a conflicting variable.
(ctf_link_one_input_archive_member): Adjust.
(ctf_link_lazy_open): New, open a CTF archive for linking when
needed.
(ctf_link_close_one_input_archive): New, close it again.
(ctf_link_one_input_archive): Adjust for lazy opening, member
renames, and ctf_err_warn transition. Move the
empty_link_type_mapping call to...
(ctf_link): ... here. Adjut for renamings and thunk removal.
Don't spuriously fail if some input contains no CTF data.
(ctf_link_write): ctf_err_warn transition.
* libctf.ver: Remove not-yet-stable comment.
Nick Alcock [Thu, 4 Jun 2020 17:07:38 +0000 (18:07 +0100)]
libctf: drop error-prone ctf_strerror
This utility function is almost useless (all it does is casts the result
of a strerror) but has a seriously confusing name. Over and over again
I have accidentally called it instead of ctf_errmsg, and hidden a
time-bomb for myself in a hard-to-test error-handling path: since
ctf_strerror is just a strerror wrapper, it cannot handle CTF errnos,
unlike ctf_errmsg. It's astonishingly lucky that none of these errors
have crept into any commits to date.
Fuse it into ctf_errmsg and drop it.
libctf/
* ctf-impl.h (ctf_strerror): Delete.
* ctf-subr.c (ctf_strerror): Likewise.
* ctf-error.c (ctf_errmsg): Stop using ctf_strerror: just use
strerror directly.
Nick Alcock [Thu, 4 Jun 2020 16:30:01 +0000 (17:30 +0100)]
libctf: sort out potential refcount loops
When you link TUs that contain conflicting types together, the resulting
CTF section is an archive containing many CTF dicts. These dicts appear
in ctf_link_outputs of the shared dict, with each ctf_import'ing that
shared dict. ctf_importing a dict bumps its refcount to stop it going
away while it's in use -- but if the shared dict (whose refcount is
bumped) has the child dict (doing the bumping) in its ctf_link_outputs,
we have a refcount loop, since the child dict only un-ctf_imports and
drops the parent's refcount when it is freed, but the child is only
freed when the parent's refcount falls to zero.
(In the future, this will be able to go wrong on the inputs too, when an
ld -r'ed deduplicated output with conflicts is relinked. Right now this
cannot happen because we don't ctf_import such dicts at all. This will
be fixed in a later commit in this series.)
Fix this by introducing an internal-use-only ctf_import_unref function
that imports a parent dict *witthout* bumping the parent's refcount, and
using it when we create per-CU outputs. This function is only safe to
use if you know the parent cannot go away while the child exists: but if
the parent *owns* the child, as here, this is necessarily true.
Record in the ctf_file_t whether a parent was imported via ctf_import or
ctf_import_unref, so that if you do another ctf_import later on (or a
ctf_import_unref) it can decide whether to drop the refcount of the
existing parent being replaced depending on which function you used to
import that one. Adjust ctf_serialize so that rather than doing a
ctf_import (which is wrong if the original import was
ctf_import_unref'fed), we just copy the parent field and refcount over
and forcibly flip the unref flag on on the old copy we are going to
discard.
ctf_file_close also needs a bit of tweaking to only close the parent if
it was not imported with ctf_import_unref: while we're at it, guard
against repeated closes with a refcount of zero and stop them causing
double-frees, even if destruction of things freed *inside*
ctf_file_close cause such recursion.
Verified no leaks or accesses to freed memory after all of this with
valgrind. (It was leak-happy before.)
libctf/
* ctf-impl.c (ctf_file_t) <ctf_parent_unreffed>: New.
(ctf_import_unref): New.
* ctf-open.c (ctf_file_close) Drop the refcount all the way to
zero. Don't recurse back in if the refcount is already zero.
(ctf_import): Check ctf_parent_unreffed before deciding whether
to close a pre-existing parent. Set it to zero.
(ctf_import_unreffed): New, as above, setting
ctf_parent_unreffed to 1.
* ctf-create.c (ctf_serialize): Do not ctf_import into the new
child: use direct assignment, and set unreffed on the new and
old children.
* ctf-link.c (ctf_create_per_cu): Import the parent using
ctf_import_unreffed.
Nick Alcock [Thu, 4 Jun 2020 16:21:10 +0000 (17:21 +0100)]
libctf: rename the type_mapping_key to type_key
The name was just annoyingly long and I kept misspelling it.
It's also a bad name: it's not a mapping the type might be *used* in a
type mapping, but it is itself a representation of a type (a ctf_file_t
/ ctf_id_t pair), not of a mapping at all.
libctf/
* ctf-impl.h (ctf_link_type_mapping_key): Rename to...
(ctf_link_type_key): ... this, adjusting member prefixes to
match.
(ctf_hash_type_mapping_key): Rename to...
(ctf_hash_type_key): ... this.
(ctf_hash_eq_type_mapping_key): Rename to...
(ctf_hash_eq_type_key): ... this.
* ctf-hash.c (ctf_hash_type_mapping_key): Rename to...
(ctf_hash_type_key): ... this, and adjust for member name
changes.
(ctf_hash_eq_type_mapping_key): Rename to...
(ctf_hash_eq_type_key): ... this, and adjust for member name
changes.
* ctf-link.c (ctf_add_type_mapping): Adjust. Note the lack of
need for out-of-memory checking in this code.
(ctf_type_mapping): Adjust.
Nick Alcock [Thu, 4 Jun 2020 16:16:09 +0000 (17:16 +0100)]
libctf: check for vasprintf
We've been using this for all of libctf's history in binutils: we should
check for it in configure.
libctf/
configure.ac: Check for vasprintf.
configure: Regenerated.
config.h.in: Likewise.
Nick Alcock [Tue, 9 Jun 2020 10:18:25 +0000 (11:18 +0100)]
libctf, archive: fix bad error message
Get the function name right.
libctf/
* ctf-archive.c (ctf_arc_bufopen): Fix message.
Nick Alcock [Tue, 9 Jun 2020 09:27:57 +0000 (10:27 +0100)]
libctf, open: fix opening CTF in binaries with no symtab
This is a perfectly possible case, and half of ctf_bfdopen_ctfsect
handled it fine. The other half hit a divide by zero or two before we
got that far, and had no code path to load the strtab from anywhere
in the absence of a symtab to point at it in any case.
So, as a fallback, if there is no symtab, try loading ".strtab"
explicitly by name, like we used to before we started looking for the
strtab the symtab used.
Of course, such a strtab is not kept hold of by BFD, so this means we
have to bring back the code to possibly explicitly free the strtab that
we read in.
libctf/
* ctf-impl.h (struct ctf_archive_internal) <ctfi_free_strsect>
New.
* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Explicitly open a strtab
if the input has no symtab, rather than dividing by
zero. Arrange to free it later via ctfi_free_ctfsect.
* ctf-archive.c (ctf_new_archive_internal): Do not
ctfi_free_strsect by default.
(ctf_arc_close): Possibly free it here.
Nick Alcock [Mon, 11 May 2020 17:18:50 +0000 (18:18 +0100)]
libctf, dump: fix slice dumping
Now that we can have slices of anything terminating in an int, we must
dump things accordingly, or slices of typedefs appear as
c5b: __u8 -> 16c: __u8 -> 78: short unsigned int (size 0x2)
which is unhelpful. If things *are* printed as slices, the name is
missing:
a15: [slice 0x8:0x4]-> 16c: __u8 -> 78: short unsigned int (size 0x2)
And struct members give no clue they're a slice at all, which is a shame
since bitfields are the major use of this type kind:
[0x8] (ID 0xa15) (kind 10) __u8 dst_reg
Fix things so that everything slicelike or integral gets its encoding
printed, and everything with a name gets the name printed:
a15: __u8 [slice 0x8:0x4] (size 0x1) -> 1ff: __u8 (size 0x1) -> 37: unsigned char [0x0:0x8] (size 0x1)
[0x0] (ID 0xa15) (kind 10) __u8:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)
Bitfield struct members get a technically redundant but much
easier-to-understand dumping now:
[0x0] (ID 0x80000005) (kind 6) struct bpf_insn (aligned at 0x1)
[0x0] (ID 0x222) (kind 10) __u8 code (aligned at 0x1)
[0x8] (ID 0x1e9e) (kind 10) __u8 dst_reg:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)
[0xc] (ID 0x1e46) (kind 10) __u8 src_reg:4 (aligned at 0x1, format 0x2, offset:bits 0xc:0x4)
[0x10] (ID 0xf35) (kind 10) __s16 off (aligned at 0x2)
[0x20] (ID 0x1718) (kind 10) __s32 imm (aligned at 0x4)
This also fixes one place where a failure to format a type would be
erroneously considered an out-of-memory condition.
libctf/
* ctf-dump.c (ctf_is_slice): Delete, unnecessary.
(ctf_dump_format_type): improve slice formatting. Always print
the type size, even of slices.
(ctf_dump_member): Print slices (-> bitfields) differently from
non-slices. Failure to format a type is not an OOM.
Nick Alcock [Thu, 4 Jun 2020 14:38:26 +0000 (15:38 +0100)]
libctf, dump: migrate towards dumping errors rather than truncation
If we get an error emitting a single type, variable, or label, right now
we emit the error into the ctf_dprintf stream and propagate the error
all the way up the stack, causing the entire output to be silently
truncated (unless libctf debugging is on).
Instead, emit an error and keep going. (This makes sense for this use
case: if you're dumping types and a type is corrupted, you want to
know!)
Not all instances of this are fixed in this commit, only ones associated
with type formatting: more fixes will come.
libctf/
* ctf-dump.c (ctf_dump_format_type): Emit a warning.
(ctf_dump_label): Swallow errors from ctf_dump_format_type.
(ctf_dump_objts): Likewise.
(ctf_dump_var): Likewise.
(ctf_dump_type): Do not emit a duplicate message. Move to
ctf_err_warning, and swallow all errors.
Nick Alcock [Tue, 9 Jun 2020 09:45:07 +0000 (10:45 +0100)]
libctf, decl: avoid leaks of the formatted string on error
ctf_decl_sprintf builds up a formatted string in the ctf_decl_t's
cd_buf, but then on error this is hardly ever freed: we assume that
ctf_decl_fini frees it, but it leaks it instead.
Make it free it like any decent ADT should.
libctf/
* ctf-decl.c (ctf_decl_fini): Free the cd_buf.
(ctf_decl_buf): Once it escapes, don't try to free it later.
Nick Alcock [Thu, 4 Jun 2020 14:25:32 +0000 (15:25 +0100)]
libctf, types: enhance ctf_type_aname to print function arg types
Somehow this never got implemented, which makes debugging any kind of
bug that has to do with argument types fantastically confusing, because
it *looks* like the func type takes no arguments though in fact it does.
This also lets us simplify the dumper slightly (and introduces our first
uses of ctf_assert and ctf_err_warn: there will be many more).
ctf_type_aname dumps function types without including the function
pointer name itself: ctf_dump search-and-replaces it in. This seems to
give the nicest-looking results for existing users of both, even if it
is a bit fiddly.
libctf/
* ctf-types.c (ctf_type_aname): Print arg types here...
* ctf-dump.c (ctf_dump_funcs): ... not here: but do substitute
in the type name here.
Nick Alcock [Thu, 4 Jun 2020 14:07:54 +0000 (15:07 +0100)]
libctf, ld, binutils: add textual error/warning reporting for libctf
This commit adds a long-missing piece of infrastructure to libctf: the
ability to report errors and warnings using all the power of printf,
rather than being restricted to one errno value. Internally, libctf
calls ctf_err_warn() to add errors and warnings to a list: a new
iterator ctf_errwarning_next() then consumes this list one by one and
hands it to the caller, which can free it. New errors and warnings are
added until the list is consumed by the caller or the ctf_file_t is
closed, so you can dump them at intervals. The caller can of course
choose to print only those warnings it wants. (I am not sure whether we
want objdump, readelf or ld to print warnings or not: right now I'm
printing them, but maybe we only want to print errors? This entirely
depends on whether warnings are voluminous things describing e.g. the
inability to emit single types because of name clashes or something.
There are no users of this infrastructure yet, so it's hard to say.)
There is no internationalization here yet, but this at least adds a
place where internationalization can be added, to one of
ctf_errwarning_next or ctf_err_warn.
We also provide a new ctf_assert() function which uses this
infrastructure to provide non-fatal assertion failures while emitting an
assert-like string to the caller: to save space and avoid needlessly
duplicating unchanging strings, the assertion test is inlined but the
print-things-out failure case is not. All assertions in libctf will be
converted to use this machinery in future commits and propagate
assertion-failure errors up, so that the linker in particular cannot be
killed by libctf assertion failures when it could perfectly well just
print warnings and drop the CTF section.
include/
* ctf-api.h (ECTF_INTERNAL): Adjust error text.
(ctf_errwarning_next): New.
libctf/
* ctf-impl.h (ctf_assert): New.
(ctf_err_warning_t): Likewise.
(ctf_file_t) <ctf_errs_warnings>: Likewise.
(ctf_err_warn): New prototype.
(ctf_assert_fail_internal): Likewise.
* ctf-inlines.h (ctf_assert_internal): Likewise.
* ctf-open.c (ctf_file_close): Free ctf_errs_warnings.
* ctf-create.c (ctf_serialize): Copy it on serialization.
* ctf-subr.c (ctf_err_warn): New, add an error/warning.
(ctf_errwarning_next): New iterator, free and pass back
errors/warnings in succession.
* libctf.ver (ctf_errwarning_next): Add.
ld/
* ldlang.c (lang_ctf_errs_warnings): New, print CTF errors
and warnings. Assert when libctf asserts.
(lang_merge_ctf): Call it.
(land_write_ctf): Likewise.
binutils/
* objdump.c (ctf_archive_member): Print CTF errors and warnings.
* readelf.c (dump_ctf_archive_member): Likewise.
Egeyar Bagcioglu [Wed, 3 Jun 2020 16:36:51 +0000 (17:36 +0100)]
libctf, types: ensure the emission of ECTF_NOPARENT
ctf_variable_iter was returning a (positive!) error code rather than
setting the error in the passed-in ctf_file_t.
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
libctf/
* ctf-types.c (ctf_variable_iter): Fix error return.
Nick Alcock [Wed, 3 Jun 2020 16:31:44 +0000 (17:31 +0100)]
libctf: error out on corrupt CTF with invalid header flags
If corrupt CTF with invalid header flags is passed in, return the new
error ECTF_FLAGS.
include/
* ctf-api.h (ECTF_FLAGS): New.
(ECTF_NERR): Adjust.
* ctf.h (CTF_F_MAX): New.
libctf/
* ctf-open.c (ctf_bufopen_internal): Diagnose invalid flags.
Nick Alcock [Tue, 30 Jun 2020 17:02:28 +0000 (18:02 +0100)]
libctf: pass the thunk down properly when wrapping qsort_r
When wrapping qsort_r on a system like FreeBSD on which the compar
argument comes first, we wrap the passed arg in a thunk so we can pass
down both the caller-supplied comparator function and its argument. We
should pass the *argument* down to the comparator, not the thunk, which
is basically random nonsense on the stack from the point of view of the
caller of qsort_r.
libctf/
ctf-decls.h (ctf_qsort_compar_thunk): Fix arg passing.
Nick Alcock [Wed, 3 Jun 2020 15:36:18 +0000 (16:36 +0100)]
libctf, next, hash: add dynhash and dynset _next iteration
This lets you iterate over dynhashes and dynsets using the _next API.
dynhashes can be iterated over in sorted order, which works by
populating an array of key/value pairs using ctf_dynhash_next itself,
then sorting it with qsort.
Convenience inline functions named ctf_dyn{hash,set}_cnext are also
provided that take (-> return) const keys and values.
libctf/
* ctf-impl.h (ctf_next_hkv_t): New, kv-pairs passed to
sorting functions.
(ctf_next_t) <u.ctn_sorted_hkv>: New, sorted kv-pairs for
ctf_dynhash_next_sorted.
<cu.ctn_h>: New, pointer to the dynhash under iteration.
<cu.ctn_s>: New, pointer to the dynset under iteration.
(ctf_hash_sort_f): Sorting function passed to...
(ctf_dynhash_next_sorted): ... this new function.
(ctf_dynhash_next): New.
(ctf_dynset_next): New.
* ctf-inlines.h (ctf_dynhash_cnext_sorted): New.
(ctf_dynhash_cnext): New.
(ctf_dynset_cnext): New.
* ctf-hash.c (ctf_dynhash_next_sorted): New.
(ctf_dynhash_next): New.
(ctf_dynset_next): New.
* ctf-util.c (ctf_next_destroy): Free the u.ctn_sorted_hkv if
needed.
(ctf_next_copy): Alloc-and-copy the u.ctn_sorted_hkv if needed.
Nick Alcock [Wed, 3 Jun 2020 14:13:24 +0000 (15:13 +0100)]
libctf, next: introduce new class of easier-to-use iterators
The libctf machinery currently only provides one way to iterate over its
data structures: ctf_*_iter functions that take a callback and an arg
and repeatedly call it.
This *works*, but if you are doing a lot of iteration it is really quite
inconvenient: you have to package up your local variables into
structures over and over again and spawn lots of little functions even
if it would be clearer in a single run of code. Look at ctf-string.c
for an extreme example of how unreadable this can get, with
three-line-long functions proliferating wildly.
The deduplicator takes this to the Nth level. It iterates over a whole
bunch of things: if we'd had to use _iter-class iterators for all of
them there would be twenty additional functions in the deduplicator
alone, for no other reason than that the iterator API requires it.
Let's do something better. strtok_r gives us half the design: generators
in a number of other languages give us the other half.
The *_next API allows you to iterate over CTF-like entities in a single
function using a normal while loop. e.g. here we are iterating over all
the types in a dict:
ctf_next_t *i = NULL;
int *hidden;
ctf_id_t id;
while ((id = ctf_type_next (fp, &i, &hidden, 1)) != CTF_ERR)
{
/* do something with 'hidden' and 'id' */
}
if (ctf_errno (fp) != ECTF_NEXT_END)
/* iteration error */
Here we are walking through the members of a struct with CTF ID
'struct_type':
ctf_next_t *i = NULL;
ssize_t offset;
const char *name;
ctf_id_t membtype;
while ((offset = ctf_member_next (fp, struct_type, &i, &name,
&membtype)) >= 0
{
/* do something with offset, name, and membtype */
}
if (ctf_errno (fp) != ECTF_NEXT_END)
/* iteration error */
Like every other while loop, this means you have access to all the local
variables outside the loop while inside it, with no need to tiresomely
package things up in structures, move the body of the loop into a
separate function, etc, as you would with an iterator taking a callback.
ctf_*_next allocates 'i' for you on first entry (when it must be NULL),
and frees and NULLs it and returns a _next-dependent flag value when the
iteration is over: the fp errno is set to ECTF_NEXT_END when the
iteartion ends normally. If you want to exit early, call
ctf_next_destroy on the iterator. You can copy iterators using
ctf_next_copy, which copies their current iteration position so you can
remember loop positions and go back to them later (or ctf_next_destroy
them if you don't need them after all).
Each _next function returns an always-likely-to-be-useful property of
the thing being iterated over, and takes pointers to parameters for the
others: with very few exceptions all those parameters can be NULLs if
you're not interested in them, so e.g. you can iterate over only the
offsets of members of a structure this way:
while ((offset = ctf_member_next (fp, struct_id, &i, NULL, NULL)) >= 0)
If you pass an iterator in use by one iteration function to another one,
you get the new error ECTF_NEXT_WRONGFUN back; if you try to change
ctf_file_t in mid-iteration, you get ECTF_NEXT_WRONGFP back.
Internally the ctf_next_t remembers the iteration function in use,
various sizes and increments useful for almost all iterations, then
uses unions to overlap the actual entities being iterated over to keep
ctf_next_t size down.
Iterators available in the public API so far (all tested in actual use
in the deduplicator):
/* Iterate over the members of a STRUCT or UNION, returning each member's
offset and optionally name and member type in turn. On end-of-iteration,
returns -1. */
ssize_t
ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
const char **name, ctf_id_t *membtype);
/* Iterate over the members of an enum TYPE, returning each enumerand's
NAME or NULL at end of iteration or error, and optionally passing
back the enumerand's integer VALue. */
const char *
ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
int *val);
/* Iterate over every type in the given CTF container (not including
parents), optionally including non-user-visible types, returning
each type ID and optionally the hidden flag in turn. Returns CTF_ERR
on end of iteration or error. */
ctf_id_t
ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag,
int want_hidden);
/* Iterate over every variable in the given CTF container, in arbitrary
order, returning the name and type of each variable in turn. The
NAME argument is not optional. Returns CTF_ERR on end of iteration
or error. */
ctf_id_t
ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name);
/* Iterate over all CTF files in an archive, returning each dict in turn as a
ctf_file_t, and NULL on error or end of iteration. It is the caller's
responsibility to close it. Parent dicts may be skipped. Regardless of
whether they are skipped or not, the caller must ctf_import the parent if
need be. */
ctf_file_t *
ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it,
const char **name, int skip_parent, int *errp);
ctf_label_next is prototyped but not implemented yet.
include/
* ctf-api.h (ECTF_NEXT_END): New error.
(ECTF_NEXT_WRONGFUN): Likewise.
(ECTF_NEXT_WRONGFP): Likewise.
(ECTF_NERR): Adjust.
(ctf_next_t): New.
(ctf_next_create): New prototype.
(ctf_next_destroy): Likewise.
(ctf_next_copy): Likewise.
(ctf_member_next): Likewise.
(ctf_enum_next): Likewise.
(ctf_type_next): Likewise.
(ctf_label_next): Likewise.
(ctf_variable_next): Likewise.
libctf/
* ctf-impl.h (ctf_next): New.
(ctf_get_dict): New prototype.
* ctf-lookup.c (ctf_get_dict): New, split out of...
(ctf_lookup_by_id): ... here.
* ctf-util.c (ctf_next_create): New.
(ctf_next_destroy): New.
(ctf_next_copy): New.
* ctf-types.c (includes): Add <assert.h>.
(ctf_member_next): New.
(ctf_enum_next): New.
(ctf_type_iter): Document the lack of iteration over parent
types.
(ctf_type_next): New.
(ctf_variable_next): New.
* ctf-archive.c (ctf_archive_next): New.
* libctf.ver: Add new public functions.
Nick Alcock [Mon, 8 Jun 2020 08:24:01 +0000 (09:24 +0100)]
libctf: add ctf_ref
This allows you to bump the refcount on a ctf_file_t, so that you can
smuggle it out of iterators which open and close the ctf_file_t for you
around the loop body (like ctf_archive_iter).
You still can't use this to preserve a ctf_file_t for longer than the
lifetime of its containing entity (e.g. ctf_archive).
include/
* ctf-api.h (ctf_ref): New.
libctf/
* libctf.ver (ctf_ref): New.
* ctf-open.c (ctf_ref): Implement it.
Nick Alcock [Wed, 3 Jun 2020 12:23:42 +0000 (13:23 +0100)]
libctf: add ctf_forwardable_kind
The internals of the deduplicator want to know if something is a type
that can have a forward to it fairly often, often enough that inlining
it brings a noticeable performance gain. Convert the one place in
libctf that can already benefit, even though it doesn't bring any sort
of performance gain there.
libctf/
* ctf-inlines.h (ctf_forwardable_kind): New.
* ctf-create.c (ctf_add_forward): Use it.
Nick Alcock [Wed, 3 Jun 2020 12:21:56 +0000 (13:21 +0100)]
libctf: move existing inlines into ctf-inlines.h
Just housekeeping.
libctf/
* ctf-impl.h (ctf_get_ctt_size): Move definition from here...
* ctf-inlines.h (ctf_get_ctt_size): ... to here.
Nick Alcock [Tue, 2 Jun 2020 21:26:38 +0000 (22:26 +0100)]
libctf, hash: introduce the ctf_dynset
There are many places in the deduplicator which use hashtables as tiny
sets: keys with no value (and usually, but not always, no freeing
function) often with only one or a few members. For each of these, even
after the last change to not store the freeing functions, we are storing
a little malloced block for each item just to track the key/value pair,
and a little malloced block for the hash table itself just to track the
freeing function because we can't use libiberty hashtab's freeing
function because we are using that to free the little malloced per-item
block.
If we only have a key, we don't need any of that: we can ditch the
per-malloced block because we don't have a value, and we can ditch the
per-hashtab structure because we don't need to independently track the
freeing functions since libiberty hashtab is doing it for us. That
means we don't need an owner field in the (now nonexistent) item block
either.
Roughly speaking, this datatype saves about 25% in time and 20% in peak
memory usage for normal links, even fairly big ones. So this might seem
redundant, but it's really worth it.
Instead of a _lookup function, a dynset has two distinct functions:
ctf_dynset_exists, which returns true or false and an optional pointer
to the set member, and ctf_dynhash_lookup_any, which is used if all
members of the set are expected to be equivalent and we just want *any*
member and we don't care which one.
There is no iterator in this set of functions, not because we don't
iterate over dynset members -- we do, a lot -- but because the iterator
here is a member of an entirely new family of much more convenient
iteration functions, introduced in the next commit.
libctf/
* ctf-hash.c (ctf_dynset_eq_string): New.
(ctf_dynset_create): New.
(DYNSET_EMPTY_ENTRY_REPLACEMENT): New.
(DYNSET_DELETED_ENTRY_REPLACEMENT): New.
(key_to_internal): New.
(internal_to_key): New.
(ctf_dynset_insert): New.
(ctf_dynset_remove): New.
(ctf_dynset_destroy): New.
(ctf_dynset_lookup): New.
(ctf_dynset_exists): New.
(ctf_dynset_lookup_any): New.
(ctf_hash_insert_type): Coding style.
(ctf_hash_define_type): Likewise.
* ctf-impl.h (ctf_dynset_t): New.
(ctf_dynset_eq_string): New.
(ctf_dynset_create): New.
(ctf_dynset_insert): New.
(ctf_dynset_remove): New.
(ctf_dynset_destroy): New.
(ctf_dynset_lookup): New.
(ctf_dynset_exists): New.
(ctf_dynset_lookup_any): New.
* ctf-inlines.h (ctf_dynset_cinsert): New.
Nick Alcock [Tue, 2 Jun 2020 21:00:14 +0000 (22:00 +0100)]
libctf, hash: save per-item space when no key/item freeing function
The libctf dynhash hashtab abstraction supports per-hashtab arbitrary
key/item freeing functions -- but it also has a constant slot type that
holds both key and value requested by the user, so it needs to use its
own freeing function to free that -- and it has nowhere to store the
freeing functions the caller requested.
So it copies them into every hash item, bloating every slot, even though
all items in a given hash table must have the same key and value freeing
functions.
So point back to the owner using a back-pointer, but don't even spend
space in the item or the hashtab allocating those freeing functions
unless necessary: if none are needed, we can simply arrange to not pass
in ctf_dynhash_item_free as a del_f to hashtab_create_alloc, and none of
those fields will ever be accessed.
The only downside is that this makes the code sensitive to the order of
fields in the ctf_helem_t and ctf_hashtab_t: but the deduplicator
allocates so many hash tables that doing this alone cuts memory usage
during deduplication by about 10%. (libiberty hashtab itself has a lot
of per-hashtab bloat: in the future we might trim that down, or make a
trimmer version.)
libctf/
* ctf-hash.c (ctf_helem_t) <key_free>: Remove.
<value_free>: Likewise.
<owner>: New.
(ctf_dynhash_item_free): Indirect through the owner.
(ctf_dynhash_create): Only pass in ctf_dynhash_item_free and
allocate space for the key_free and value_free fields fields
if necessary.
(ctf_hashtab_insert): Likewise. Fix OOM errno value.
(ctf_dynhash_insert): Only access ctf_hashtab's key_free and
value_free if they will exist. Set the slot's owner, but only
if it exists.
(ctf_dynhash_remove): Adjust.
Nick Alcock [Tue, 2 Jun 2020 20:48:12 +0000 (21:48 +0100)]
libctf, hash: improve insertion of existing keys into dynhashes
Right now, if you insert a key/value pair into a dynhash, the old slot's
key is freed and the new one always assigned. This seemed sane to me
when I wrote it, but I got it wrong time and time again. It's much
less confusing to free the key passed in: if a key-freeing function
was passed, you are asserting that the dynhash owns the key in any
case, so if you pass in a key it is always buggy to assume it sticks
around. Freeing the old key means that you can't even safely look up a
key from out of a dynhash and hold on to it, because some other matching
key might force it to be freed at any time.
In the new model, you can always get a key out of a dynhash with
ctf_dynhash_lookup_kv and hang on to it until the kv-pair is actually
deleted from the dynhash. In the old model the pointer to the key might
be freed at any time if a matching key was inserted.
libctf/
* ctf-hash.c (ctf_hashtab_insert): Free the key passed in if
there is a key-freeing function and the key already exists.
Nick Alcock [Tue, 2 Jun 2020 20:31:45 +0000 (21:31 +0100)]
libctf: add new dynhash functions
Future commits will use these.
ctf_dynhash_elements: count elements in a dynhash
ctf_dynhash_lookup_kv: look up and return pointers to the original key
and value in a dynhash (the only way of getting
a reference to the original key)
ctf_dynhash_iter_find: iterate until an item is found, then return its
key
ctf_dynhash_cinsert: insert a const key / value into a dynhash (a thim
wrapper in a new header dedicated to inline
functions).
As with the rest of ctf_dynhash, this is not public API. No impact
on existing callers is expected.
libctf/
* ctf-inlines.h: New file.
* ctf-impl.h: Include it.
(ctf_hash_iter_find_f): New typedef.
(ctf_dynhash_elements): New.
(ctf_dynhash_lookup_kv): New.
(ctf_dynhash_iter_find): New.
* ctf-hash.c (ctf_dynhash_lookup_kv): New.
(ctf_traverse_find_cb_arg_t): New.
(ctf_hashtab_traverse_find): New.
(ctf_dynhash_iter_find): New.
(ctf_dynhash_elements): New.
Nick Alcock [Tue, 2 Jun 2020 20:16:15 +0000 (21:16 +0100)]
libctf: fix __extension__ with non-GNU C compilers
We forgot to #define __extension__ to nothing in this case.
libctf/
* ctf-impl.h [!__GNUC__] (__extension__): Define to nothing.
Nick Alcock [Tue, 2 Jun 2020 20:14:22 +0000 (21:14 +0100)]
libctf: add ctf_archive_count
Another count that was otherwise unavailable without doing expensive
operations.
include/
* ctf-api.h (ctf_archive_count): New.
libctf/
* ctf-archive.c (ctf_archive_count): New.
* libctf.ver: New public function.
Nick Alcock [Tue, 2 Jun 2020 20:11:25 +0000 (21:11 +0100)]
libctf: add ctf_member_count
This returns the number of members in a struct or union, or the number
of enumerations in an enum. (This was only available before now by
iterating across every member, but it can be returned much faster than
that.)
include/
* ctf-api.h (ctf_member_count): New.
libctf/
* ctf-types.c (ctf_member_count): New.
* libctf.ver: New public function.
Nick Alcock [Tue, 2 Jun 2020 20:09:49 +0000 (21:09 +0100)]
libctf: add ctf_type_kind_forwarded
This is just like ctf_type_kind, except that forwards get the
type of the thing being pointed to rather than CTF_K_FORWARD.
include/
* ctf-api.h (ctf_type_kind_forwarded): New.
libctf/
* ctf-types.c (ctf_type_kind_forwarded): New.
Nick Alcock [Tue, 2 Jun 2020 20:06:18 +0000 (21:06 +0100)]
libctf: add ctf_type_name_raw
We already have a function ctf_type_aname_raw, which returns the raw
name of a type with no decoration for structures or arrays or anything
like that: just the underlying name of whatever it is that's being
ultimately pointed at.
But this can be inconvenient to use, becauswe it always allocates new
storage for the string and copies it in, so it can potentially fail.
Add ctf_type_name_raw, which just returns the string directly out of
libctf's guts: it will live until the ctf_file_t is closed (if we later
gain the ability to remove types from writable dicts, it will live as
long as the type lives).
Reimplement ctf_type_aname_raw in terms of it.
include/
* ctf-api.c (ctf_type_name_raw): New.
libctf/
* ctf-types.c (ctf_type_name_raw): New.
(ctf_type_aname_raw): Reimplement accordingly.
Nick Alcock [Tue, 2 Jun 2020 20:00:35 +0000 (21:00 +0100)]
libctf: having debugging enabled is unlikely
The deduplicator can emit enormous amounts of debugging output,
so much so that a later commit will introduce a new configure flag
that configures most of it out (and configures it out by default).
It became clear that when this configure flag is on, but debugging is
not enabled via the LIBCTF_DEBUG environment variable, up to 10% of
runtime can be spent on branch mispredictions checking the _libctf_debug
variable. Mark it unlikely to be set (when it is set, performance is
likely to be the least of your concerns).
libctf/
* ctf-subr.c (ctf_dprintf): _libctf_debug is unlikely to be set.
Nick Alcock [Tue, 2 Jun 2020 19:55:05 +0000 (20:55 +0100)]
libctf, archive: stop ctf_arc_bufopen triggering crazy unmaps
The archive machinery mmap()s its archives when possible: so it arranges
to do appropriately-sized unmaps by recording the unmap length in the
ctfa_magic value and unmapping that.
This brilliant (horrible) trick works less well when ctf_arc_bufopen is
called with an existing buffer (which might be a readonly mapping).
ctf_arc_bufopen always returns a ctf_archive_t wrapper, so record in
there the necessity to not unmap anything when a bufopen'ed archive is
closed again.
libctf/
* ctf-impl.h (struct ctf_archive_internal)
<ctfi_unmap_on_close>: New.
(ctf_new_archive_internal): Adjust.
* ctf-archive.c (ctf_new_archive_internal): Likewise.
Initialize ctfi_unmap_on_close. Adjust error path.
(ctf_arc_bufopen): Adjust ctf_new_archive_internal call
(unmap_on_close is 0).
(ctf_arc_close): Only unmap if ctfi_unmap_on_close.
* ctf-open-bfd.c (ctf_fdopen): Adjust.
Nick Alcock [Tue, 2 Jun 2020 19:49:14 +0000 (20:49 +0100)]
libctf, types: ints, floats and typedefs with no name are invalid
Report them as such, rather than letting ctf_decl_sprintf wrongly
conclude that the printing of zero characters means we are out of
memory.
libctf/
* ctf-types.c (ctf_type_aname): Return ECTF_CORRUPT if
ints, floats or typedefs have no name. Fix comment typo.
Nick Alcock [Tue, 2 Jun 2020 19:43:03 +0000 (20:43 +0100)]
libctf, types: support slices of anything terminating in an int
It is perfectly valid C to say e.g.
typedef u64 int;
struct foo_t
{
const volatile u64 wibble:2;
};
i.e. bitfields have to be integral types, but they can be cv-qualified
integral types or typedefs of same, etc.
This is easy to fix: do a ctf_type_resolve_unsliced() at creation time
to ensure the ultimate type is integral, and ctf_type_resolve() at
lookup time so that if you somehow have e.g. a slice of a typedef of a
slice of a cv-qualified int, we pull the encoding that the topmost slice
is based on out of the subsidiary slice (and then modify it), not out of
the underlying int. (This last bit is rather academic right now, since
all slices override exactly the same properties of the underlying type,
but it's still the right thing to do.)
libctf/
* ctf-create.c (ctf_add_slice): Support slices of any kind that
resolves to an integral type.
* ctf-types.c (ctf_type_encoding): Resolve the type before
fishing its encoding out.
Nick Alcock [Tue, 2 Jun 2020 19:39:15 +0000 (20:39 +0100)]
libctf, create: empty dicts are dirty to start with
Without this, an empty dict that is written out immediately never gets
any content at all: even the header is left empty.
libctf/
* ctf-create.c (ctf_create): Mark dirty.
Nick Alcock [Tue, 2 Jun 2020 19:38:17 +0000 (20:38 +0100)]
libctf, create: fix addition of anonymous struct/union members
A Solaris-era bug causes us to check the offsets of types with no names
against the first such type when ctf_add_type()ing members to a struct
or union. Members with no names (i.e. anonymous struct/union members)
can appear as many times as you like in a struct/union, so this check
should be skipped in this case.
libctf/
* ctf-create.c (membcmp) Skip nameless members.
Nick Alcock [Tue, 2 Jun 2020 19:28:16 +0000 (20:28 +0100)]
libctf, create: member names of "" and NULL should be the same
This matters for the case of unnamed bitfields, whose names are the null
string. These are special in that they are the only members whose
"names" are allowed to be duplicated in a single struct, but we were
only handling this for the case where name == NULL. Translate "" to
NULL to help callers.
libctf/
* ctf-create.c (ctf_add_member_offset): Support names of ""
as if they were the null pointer.
Nick Alcock [Tue, 2 Jun 2020 19:26:29 +0000 (20:26 +0100)]
libctf, open: drop unnecessary historical wart around forwards
When opening, we consider a forward with a kind above the maximum
allowable set of kinds and a forward of kind CTF_K_UNKNOWN to be a
forward to a struct. Whatever CTF version it was that produced
forwards with no associated kind, it predates anything we can read:
remove this wart.
libctf/
* ctf-open.c (init_types): Remove typeless CTF_K_FORWARD
special-casing.
Nick Alcock [Tue, 2 Jun 2020 19:25:02 +0000 (20:25 +0100)]
libctf, types: allow ctf_type_reference of dynamic slices
One spot was missed when we rejigged ctf_update into ctf_serialize and
allowed all operations on dynamic containers: ctf_type_reference of
slices. A dynamic slice's vlen state is stored in the dtu_slice member,
so fetch it from there.
libctf/
* ctf-types.c (ctf_type_reference): Add support for dynamic slices.
Nick Alcock [Tue, 2 Jun 2020 19:23:07 +0000 (20:23 +0100)]
libctf, create: add explicit casts for variables' and slices' types
This is technically unnecessary -- the compiler is quite capable of
doing the range reduction for us -- but it does mean that all
assignments of a ctf_id_t to its final uint32_t representation now have
appropriate explicit casts.
libctf/
* ctf-create.c (ctf_serialize): Add cast.
(ctf_add_slice): Likewise.
Nick Alcock [Tue, 2 Jun 2020 19:17:51 +0000 (20:17 +0100)]
libctf, create: do not corrupt function types' arglists at insertion time
ctf_add_function assumes that function types' arglists are of type
ctf_id_t. Since they are CTF IDs, they are 32 bits wide, a uint32_t:
unfortunately ctf_id_t is a forward-compatible user-facing 64 bits wide,
and should never ever reach the CTF storage level.
All the CTF code other than ctf_add_function correctly assumes that
function arglists outside dynamic containers are 32 bits wide, so the
serialization machinery ends up cutting off half the arglist, corrupting
all args but the first (a good sign is a bunch of args of ID 0, the
unimplemented type, popping up).
Fix this by copying the arglist into place item by item, casting it
properly, at the same time as we validate the arg types. Fix the type
of the dtu_argv in the dynamic container and drop the now-unnecessary
cast in the serializer.
libctf/
* ctf-impl.h (ctf_dtdef_t) <dtu_argv>: Fix type.
* ctf-create.c (ctf_add_function): Check for unimplemented type
and populate at the same time. Populate one-by-one, not via
memcpy.
(ctf_serialize): Remove unnecessary cast.
* ctf-types.c (ctf_func_type_info): Likewise.
(ctf_func_type_args): Likewise. Fix comment typo.
Nick Alcock [Tue, 2 Jun 2020 19:04:24 +0000 (20:04 +0100)]
libctf, create: support addition of references to the unimplemented type
The deduplicating linker adds types from the linker inputs to the output
via the same API everyone else does, so it's important that we can emit
everything that the compiler wants us to. Unfortunately, the compiler
may represent the unimplemented type (used for compiler constructs that
CTF cannot currently encode) as type zero or as a type of kind
CTF_K_UNKNOWN, and we don't allow the addition of types that cite the
former.
Adding this support adds a tiny bit of extra complexity: additions of
structure members immediately following a member of the unimplemented
type must be via ctf_add_member_offset or ctf_add_member_encoded, since
we have no idea how big members of the unimplemented type are.
(Attempts to do otherwise return -ECTF_NONREPRESENTABLE, like other
attempts to do forbidden things with the unimplemented type.)
Even slices of the unimplemented type are permitted: this is the only
case in which you can slice a type that terminates in a non-integral
type, on the grounds that it was likely integral in the source code,
it's just that we can't represent that sort of integral type properly
yet.
libctf/
* ctf-create.c (ctf_add_reftype): Support refs to type zero.
(ctf_add_array): Support array contents of type zero.
(ctf_add_function): Support arguments and return types of
type zero.
(ctf_add_typedef): Support typedefs to type zero.
(ctf_add_member_offset): Support members of type zero,
unless added at unspecified (naturally-aligned) offset.
Nick Alcock [Tue, 2 Jun 2020 18:07:08 +0000 (19:07 +0100)]
libctf: restructure error handling to reduce relocations
Jose Marchesi noted that the traditional-Unix error array in ctf-error.c
introduces one reloc per error to initialize the array: 58 so far. We
can reduce this to zero using an array of carefully-sized individual
members which is used to construct a string table, that is then
referenced by the lookup functions: but doing this automatically is a
pain.
Bruno Haible wrote suitable code years ago: I got permission to reuse it
(Bruno says "... which I hereby put in the public domain"); I modified
it a tiny bit (similarly to what Ulrich Drepper did in the dsohowto
text, but I redid it from scratch), commented it up a bit, and shifted
the error table into that form, migrating it into the new file
ctf-error.h.
This has the advantage that it spotted both typos in the text of the
errors in the comments in ctf-api.h and typos in the error defines in
the comments in ctf-error.c, and places where the two were simply not
in sync. All are now fixed.
One new constant exists in ctf-api.h: CTF_NERR, since the old method of
working out the number of errors in ctf-error.c was no longer usable,
and it seems that the number of CTF errors is something users might
reasonably want as well. It should be pretty easy to keep up to date as
new errors are introduced.
include/
* ctf-api.h (ECTF_*): Improve comments.
(ECTF_NERR): New.
libctf/
* ctf-error.c: Include <stddef.h>, for offsetof.
(_ctf_errlist): Migrate to...
(_ctf_errlist_t): ... this.
(_ctf_erridx): New, indexes into _ctf_errlist_t.
(_ctf_nerr): Remove.
(ctf_errmsg): Adjust accordingly.
* Makefile.am (BUILT_SOURCES): Note...
(ctf-error.h): ... this new rule.
* Makefile.in: Regenerate.
* mkerrors.sed: New, process ctf-api.h to generate ctf-error.h.
* .gitignore: New, ignore ctf-error.h.
Nick Alcock [Tue, 2 Jun 2020 17:56:06 +0000 (18:56 +0100)]
include, libctf: typo fixes
include/
* ctf-api.h: Fix typos in comments.
libctf/
* ctf-impl.h: Fix typos in comments.
Reuben Thomas [Wed, 22 Jul 2020 15:15:29 +0000 (16:15 +0100)]
Correct an error in the remote protocol specification
The list of commands that a stub must implement was wrong.
gdb/ChangeLog:
2020-07-22 Reuben Thomas <rrt@sc3d.org>
* gdb.texinfo (Remote Protocol, Overview): Correct the description
of which remote protocol commands are mandatory for a stub to
implement.
Andrew Burgess [Wed, 22 Jul 2020 13:57:55 +0000 (14:57 +0100)]
gdb/python: Use reference not pointer in py-registers.c
Pedro's review comments arrived after I'd already committed this
change:
commit
f7306dac19c502232f766c3881313857915f330d
Date: Tue Jul 7 15:00:30 2020 +0100
gdb/python: Reuse gdb.RegisterDescriptor objects where possible
See:
https://sourceware.org/pipermail/gdb-patches/2020-July/170726.html
There should be no user visible changes after this commit.
gdb/ChangeLog:
* python/py-registers.c (gdbpy_register_object_data_init): Remove
redundant local variable.
(gdbpy_get_register_descriptor): Extract descriptor vector as a
reference, not pointer, update code accordingly.
Nick Clifton [Wed, 22 Jul 2020 15:07:48 +0000 (16:07 +0100)]
Fix problems in CTF handling code exposed by the Coverity static analysis tool.
readelf * readelf.c (parse_args): Silence potential warnings about a
memory resource leak when allocating space for ctf option values.
(dump_section_as_ctf): Fix typo checking dump_ctf_strtab_name
variable.
libctf * ctf-archive.c (ctf_arc_write): Avoid calling close twice on the
same file descriptor.
Simon Marchi [Wed, 22 Jul 2020 13:56:08 +0000 (15:56 +0200)]
gdb/jit: skip jit symbol lookup if already detected the symbols don't exist
To detect whether an objfile is a JITer, we lookup JIT interface
symbols in the objfile. If an objfile does not have these symbols, we
conclude that it is not a JITer. An objfile that does not have the
symbols will never have them. Therefore, once we do a lookup and find
out that the objfile does not have JIT symbols, just set a flag so
that we can skip symbol lookup for that objfile the next time we reset
JIT breakpoints.
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
* objfiles.h (struct objfile) <skip_jit_symbol_lookup>: New field.
* jit.c (jit_breakpoint_re_set_internal): Use the
`skip_jit_symbol_lookup` field.
Simon Marchi [Wed, 22 Jul 2020 13:56:08 +0000 (15:56 +0200)]
gdb/jit: apply minor cleanup and modernization
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
* jit.c (jit_read_descriptor): Define the descriptor address once,
use twice.
(jit_breakpoint_deleted): Move the declaration of the loop variable
`iter` into the loop header.
(jit_breakpoint_re_set_internal): Move the declaration of the local
variable `objf_data` to the first point of definition.
(jit_event_handler): Move the declaration of local variables
`code_entry`, `entry_addr`, and `objf` to their first point of use.
Rename `objf` to `jited`.
Simon Marchi [Wed, 22 Jul 2020 13:56:08 +0000 (15:56 +0200)]
gdb/jit: remove jiter_objfile_data -> objfile back-link
This is no longer needed, remove it.
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
* jit.h (struct jiter_objfile_data) <jiter_objfile_data, objfile>:
Remove.
* jit.c (get_jiter_objfile_data): Update.
Tankut Baris Aktemur [Wed, 22 Jul 2020 13:56:07 +0000 (15:56 +0200)]
gdb/jit: enable tracking multiple JITer objfiles
GDB's JIT handler stores an objfile (and data associated with it) per
program space to keep track of JIT breakpoint information. This assumes
that there is at most one JITer objfile in the program space. However,
there may be multiple. If so, only the first JITer's hook breakpoints
would be realized and the JIT events from the other JITers would be
missed.
This patch removes that assumption, allowing an arbitrary number of
objfiles within a program space to be JITers.
- The "unique" program_space -> JITer objfile pointer in
jit_program_space_data is removed. In fact, jit_program_space_data
becomes empty, so it is removed entirely.
- jit_breakpoint_deleted is modified, it now has to assume that any
objfile in a program space is a potential JITer. It now iterates on
all objfiles, checking if they are indeed JITers, and if they are,
whether the deleted breakpoint belongs to them.
- jit_breakpoint_re_set_internal also has to assume that any objfile in
a program space is a potential JITer. It creates (or updates) one
jiter_objfile_data structure for each JITer it finds.
- Same for jit_inferior_init. It now iterates all objfiles to read the
initial JIT object list.
gdb/ChangeLog:
2020-07-22 Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
Simon Marchi <simon.marchi@polymtl.ca>
* jit.c (struct jit_program_space_data): Remove.
(jit_program_space_key): Remove.
(jiter_objfile_data::~jiter_objfile_data): Remove program space
stuff.
(get_jit_program_space_data): Remove.
(jit_breakpoint_deleted): Iterate on all of the program space's
objfiles.
(jit_inferior_init): Likewise.
(jit_breakpoint_re_set_internal): Likewise. Also change return
type to void.
(jit_breakpoint_re_set): Pass current_program_space to
jit_breakpoint_re_set_internal.
gdb/testsuite/ChangeLog:
2020-07-22 Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
* gdb.base/jit-reader-simple.exp: Add a scenario for a binary that
loads two JITers.
Simon Marchi [Wed, 22 Jul 2020 13:56:07 +0000 (15:56 +0200)]
gdb/jit: move cached_code_address and jit_breakpoint to jiter_objfile_data
This is in preparation for allowing more than one JITer objfile per
program space. Once we do that, each JITer objfile will have its own
JIT breakpoint (on the __jit_debug_register_code function it provides).
The cached_code_address field is just the runtime / relocated address of
that symbol.
Since they are going to become JITer-objfile-specific and not
program-space-specific, move these fields from jit_program_space_data to
jiter_objfile_data.
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
* jit.h (struct jiter_objfile_data) <cached_code_address,
jit_breakpoint>: Move to here from ...
* jit.c (jit_program_space_data): ... here.
(jiter_objfile_data::~jiter_objfile_data): Update.
(jit_breakpoint_deleted): Update.
(jit_breakpoint_re_set_internal): Update.
Simon Marchi [Wed, 22 Jul 2020 13:56:07 +0000 (15:56 +0200)]
gdb/jit: apply some simplifications and assertions
Following patch "gdb/jit: split jit_objfile_data in two", there are some
simplifications we can make. The invariants described there mean that
we can assume / assert some things instead of checking them using
conditionals.
If an instance of jiter_objfile_data exists for a given objfile, it's
because the required JIT interface symbols were found. Therefore, in
~jiter_objfile_data, the `register_code` field can't be NULL. It was
previously used to differentiate a jit_objfile_data object used for a
JITer vs a JITed. We can remove that check.
If an instance of jiter_objfile_data exists for a given objfile, it's
because it's the sole JITer objfile in the scope of its program space
(jit_program_space_data::objfile points to it). At the moment,
jit_breakpoint_re_set_internal won't create a second instance of
jiter_objfile_data for a given program space. Therefore, it's not
necessary to check for `ps_data != NULL` in ~jiter_objfile_data: we know
a jit_program_space_data for that program space exists. We also don't
need to check for `ps_data->objfile == this->objfile`, because we know
the objfile is the sole JITer in this program space. Replace these two
conditions with assertions.
A pre-condition for calling the jit_read_descriptor function (which is
respected in the two call sites) is that the objfile `jiter` _is_ a
JITer - it already has a jiter_objfile_data attached to it. When a
jiter_objfile_data exists, its `descriptor` field is necessarily set:
had the descriptor symbol not been found, jit_breakpoint_re_set_internal
would not have created the jiter_objfile_data. Remove the check and
early return in jit_read_descriptor. Access objfile's `jiter_data` field
directly instead of calling `get_jiter_objfile_data` (which creates the
jiter_objfile_data if it doesn't exist yet) and assert that the result
is not nullptr.
Finally, `jit_event_handler` is always passed a JITer objfile. So, add
an assertion to ensure that.
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
* jit.c (jiter_objfile_data::~jiter_objfile_data): Remove some
checks.
(jit_read_descriptor): Remove NULL check.
(jit_event_handler): Add an assertion.
Simon Marchi [Wed, 22 Jul 2020 13:56:07 +0000 (15:56 +0200)]
gdb/jit: split jit_objfile_data in two
The jit_objfile_data is currently used to hold information about both
objfiles that are the result of JIT compilation (JITed) and objfiles
that can produce JITed objfiles (JITers). I think that this double use
of the type is confusing, and that things would be more obvious if we
had one type for each role.
This patch splits it into:
- jited_objfile_data: for data about an objfile that is the result of a
JIT compilation
- jiter_objfile_data: for data about an objfile which produces JITed
objfiles
There are now two JIT-related fields in an objfile, one for each kind.
With this change, the following invariants hold:
- an objfile has a non-null `jiter_data` field iff it defines the required
symbols of the JIT interface
- an objfile has a non-null `jited_data` field iff it is the product of
JIT compilation (has been produced by some JITer)
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
* jit.h (struct jit_objfile_data): Split into...
(struct jiter_objfile_data): ... this ...
(struct jited_objfile_data): ... and this.
* objfiles.h (struct objfile) <jit_data>: Remove.
<jiter_data, jited_data>: New fields.
* jit.c (jit_objfile_data::~jit_objfile_data): Rename to ...
(jiter_objfile_data::~jiter_objfile_data): ... this.
(get_jit_objfile_data): Rename to ...
(get_jiter_objfile_data): ... this.
(add_objfile_entry): Update.
(jit_read_descriptor): Use get_jiter_objfile_data.
(jit_find_objf_with_entry_addr): Use objfile's jited_data field.
(jit_breakpoint_re_set_internal): Use get_jiter_objfile_data.
(jit_inferior_exit_hook): Use objfile's jited_data field.
Simon Marchi [Wed, 22 Jul 2020 13:56:06 +0000 (15:56 +0200)]
gdb/jit: link to jit_objfile_data directly from the objfile struct
Remove the use of objfile_data to associate a jit_objfile_data with an
objfile. Instead, directly link to a jit_objfile_data from an objfile
struct. The goal is to eliminate unnecessary abstraction.
The free_objfile_data function naturally becomes the destructor of
jit_objfile_data. However, free_objfile_data accesses the objfile to
which the data is attached, which the destructor of jit_objfile_data
doesn't have access to. To work around this, add a backlink to the
owning objfile in jit_objfile_data. This is however temporary, it goes
away in a subsequent patch.
gdb/ChangeLog:
2020-07-22 Simon Marchi <simon.marchi@polymtl.ca>
* jit.h: Forward-declare `struct minimal_symbol`.
(struct jit_objfile_data): Migrate to here from jit.c; also add a
constructor, destructor, and an objfile* field.
* jit.c (jit_objfile_data): Remove.
(struct jit_objfile_data): Migrate from here to jit.h.
(jit_objfile_data::~jit_objfile_data): New destructor
implementation with code moved from free_objfile_data.
(free_objfile_data): Delete.
(get_jit_objfile_data): Update to use the jit_data field of objfile.
(jit_find_objf_with_entry_addr): Ditto.
(jit_inferior_exit_hook): Ditto.
(_initialize_jit): Remove the call to
register_objfile_data_with_cleanup.
* objfiles.h (struct objfile) <jit_data>: New field.
Tankut Baris Aktemur [Wed, 22 Jul 2020 13:56:06 +0000 (15:56 +0200)]
gdb/jit: pass the jiter objfile as an argument to jit_event_handler
This is a refactoring that adds a new parameter to the `jit_event_handler`
function: the JITer objfile. The goal is to distinguish which JITer
triggered the JIT event, in case there are multiple JITers -- a capability
that is added in a subsequent patch.
gdb/ChangeLog:
2020-07-22 Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
* jit.h: Forward-declare `struct objfile`.
(jit_event_handler): Add a second parameter, the JITer objfile.
* jit.c (jit_read_descriptor): Change the signature to take the
JITer objfile as an argument instead of the jit_program_space_data.
(jit_inferior_init): Update the call to jit_read_descriptor.
(jit_event_handler): Use the new JITer objfile argument when calling
jit_read_descriptor.
* breakpoint.c (handle_jit_event): Update the call to
jit_event_handler to pass the JITer objfile.
Maciej W. Rozycki [Wed, 22 Jul 2020 11:46:00 +0000 (12:46 +0100)]
MIPS/GAS/testsuite: Fix JALR relocation tests for IRIX targets
With IRIX targets the JALR hint relocation is not produced for the o32
ABI, where it is considered a GNU extension. Consequently several tests
fail as the output produced by GAS fails to match patterns expecting the
relocation to be present where appropriate, even though output produced
is indeed correct.
As the absence of the relocation is expected, fix the tests by providing
respective alternative dump patterns with any JALR relocations removed,
removing numerous failures with `*-*-irix*' targets:
FAIL: MIPS jal-svr4pic (interaptiv-mr2)
FAIL: MIPS jal-svr4pic (micromips)
FAIL: MIPS jal-svr4pic (mips1)
FAIL: MIPS jal-svr4pic (mips2)
FAIL: MIPS jal-svr4pic (mips3)
FAIL: MIPS jal-svr4pic (mips4)
FAIL: MIPS jal-svr4pic (mips5)
FAIL: MIPS jal-svr4pic (mips32)
FAIL: MIPS jal-svr4pic (mips32r2)
FAIL: MIPS jal-svr4pic (mips32r3)
FAIL: MIPS jal-svr4pic (mips32r5)
FAIL: MIPS jal-svr4pic (mips32r6)
FAIL: MIPS jal-svr4pic (mips64)
FAIL: MIPS jal-svr4pic (mips64r2)
FAIL: MIPS jal-svr4pic (mips64r3)
FAIL: MIPS jal-svr4pic (mips64r5)
FAIL: MIPS jal-svr4pic (mips64r6)
FAIL: MIPS jal-svr4pic (octeon)
FAIL: MIPS jal-svr4pic (octeon2)
FAIL: MIPS jal-svr4pic (octeon3)
FAIL: MIPS jal-svr4pic (octeonp)
FAIL: MIPS jal-svr4pic (r3000)
FAIL: MIPS jal-svr4pic (r3900)
FAIL: MIPS jal-svr4pic (r4000)
FAIL: MIPS jal-svr4pic (r5900)
FAIL: MIPS jal-svr4pic (sb1)
FAIL: MIPS jal-svr4pic (vr5400)
FAIL: MIPS jal-svr4pic (xlr)
FAIL: MIPS jal-svr4pic noreorder (interaptiv-mr2)
FAIL: MIPS jal-svr4pic noreorder (micromips)
FAIL: MIPS jal-svr4pic noreorder (mips1)
FAIL: MIPS jal-svr4pic noreorder (mips2)
FAIL: MIPS jal-svr4pic noreorder (mips3)
FAIL: MIPS jal-svr4pic noreorder (mips4)
FAIL: MIPS jal-svr4pic noreorder (mips5)
FAIL: MIPS jal-svr4pic noreorder (mips32)
FAIL: MIPS jal-svr4pic noreorder (mips32r2)
FAIL: MIPS jal-svr4pic noreorder (mips32r3)
FAIL: MIPS jal-svr4pic noreorder (mips32r5)
FAIL: MIPS jal-svr4pic noreorder (mips32r6)
FAIL: MIPS jal-svr4pic noreorder (mips64)
FAIL: MIPS jal-svr4pic noreorder (mips64r2)
FAIL: MIPS jal-svr4pic noreorder (mips64r3)
FAIL: MIPS jal-svr4pic noreorder (mips64r5)
FAIL: MIPS jal-svr4pic noreorder (mips64r6)
FAIL: MIPS jal-svr4pic noreorder (octeon)
FAIL: MIPS jal-svr4pic noreorder (octeon2)
FAIL: MIPS jal-svr4pic noreorder (octeon3)
FAIL: MIPS jal-svr4pic noreorder (octeonp)
FAIL: MIPS jal-svr4pic noreorder (r3000)
FAIL: MIPS jal-svr4pic noreorder (r3900)
FAIL: MIPS jal-svr4pic noreorder (r4000)
FAIL: MIPS jal-svr4pic noreorder (r5900)
FAIL: MIPS jal-svr4pic noreorder (sb1)
FAIL: MIPS jal-svr4pic noreorder (vr5400)
FAIL: MIPS jal-svr4pic noreorder (xlr)
FAIL: MIPS R3000 jal-xgot
FAIL: MIPS -mabi=32 test 2 (SVR4 PIC)
FAIL: gas/mips/jalr2
FAIL: Relax microMIPS branches (pic)
FAIL: Relax microMIPS branches (insn32 mode, pic)
Strictly speaking no MIPSr6 or microMIPS target is supported by IRIX,
but GAS supports such configurations on the basis of uniformity, so
provide the relevant patterns too rather than excluding the combinations
from testing.
gas/
* testsuite/gas/mips/jal-svr4pic-irix.d: New file.
* testsuite/gas/mips/mips1@jal-svr4pic-irix.d: New file.
* testsuite/gas/mips/mipsr6@jal-svr4pic-irix.d: New file.
* testsuite/gas/mips/micromips@jal-svr4pic-irix.d: New file.
* testsuite/gas/mips/r3000@jal-svr4pic-irix.d: New file.
* testsuite/gas/mips/jal-svr4pic-local-irix.d: New file.
* testsuite/gas/mips/mips1@jal-svr4pic-local-irix.d: New file.
* testsuite/gas/mips/micromips@jal-svr4pic-local-irix.d: New
file.
* testsuite/gas/mips/r3000@jal-svr4pic-local-irix.d: New file.
* testsuite/gas/mips/jal-svr4pic-noreorder-irix.d: New file.
* testsuite/gas/mips/mips1@jal-svr4pic-noreorder-irix.d: New
file.
* testsuite/gas/mips/mipsr6@jal-svr4pic-noreorder-irix.d: New
file.
* testsuite/gas/mips/micromips@jal-svr4pic-noreorder-irix.d: New
file.
* testsuite/gas/mips/r3000@jal-svr4pic-noreorder-irix.d: New
file.
* testsuite/gas/mips/jal-xgot-irix.d: New file.
* testsuite/gas/mips/jalr2-irix.d: New file.
* testsuite/gas/mips/micromips-branch-relax-insn32-pic-irix.d:
New file.
* testsuite/gas/mips/micromips-branch-relax-pic-irix.d: New
file.
* testsuite/gas/mips/mips-abi32-pic2-irix.d: New file.
* testsuite/gas/mips/jal-svr4pic-local.d: Don't exclude
`*-*-irix*' targets. Add source file designator.
* testsuite/gas/mips/mips1@jal-svr4pic-local.d: Don't exclude
`*-*-irix*' targets.
* testsuite/gas/mips/r3000@jal-svr4pic-local.d: Likewise.
* testsuite/gas/mips/micromips@jal-svr4pic-local.d: Likewise.
* testsuite/gas/mips/jalr2.d: Add name designator.
* testsuite/gas/mips/mips.exp: Use respective IRIX variants for
tests involving the JALR relocation throughout.
Maciej W. Rozycki [Wed, 22 Jul 2020 11:46:00 +0000 (12:46 +0100)]
MIPS/GAS/testsuite: Use a helper variable for IRIX/non-IRIX test selection
Define a helper variable for IRIX/non-IRIX test selection and use it
with the PR 14798 test case.
gas/
* testsuite/gas/mips/mips.exp: Use a helper variable for
IRIX/non-IRIX test selection.
Pedro Alves [Wed, 22 Jul 2020 11:32:53 +0000 (12:32 +0100)]
gdbserver: handle running threads in qXfer:threads:read
On some systems, the gdb.multi/multi-target.exp testcase occasionally
fails like so:
Running src/gdb/testsuite/gdb.multi/multi-target.exp ...
FAIL: gdb.multi/multi-target.exp: info-inferiors: multi_process=on: inferior 1: info connections
FAIL: gdb.multi/multi-target.exp: info-inferiors: multi_process=on: inferior 1: info inferiors
FAIL: gdb.multi/multi-target.exp: info-inferiors: multi_process=on: inferior 2: info connections
FAIL: gdb.multi/multi-target.exp: info-inferiors: multi_process=on: inferior 2: info inferiors
FAIL: gdb.multi/multi-target.exp: info-inferiors: multi_process=on: inferior 3: inferior 3
... many more cascading fails.
The problem starts when the testcase runs an inferior against GDBserver:
(gdb) run
Starting program: build/gdb/testsuite/outputs/gdb.multi/multi-target/multi-target
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading /lib64/ld-2.31.so from remote target...
Reading /lib64/.debug/ld-2.31.so from remote target...
Reading /usr/lib/debug//lib64/ld-2.31.so from remote target...
Reading /usr/lib/debug/lib64//ld-2.31.so from remote target...
Reading target:/usr/lib/debug/lib64//ld-2.31.so from remote target...
Reading /lib/x86_64-linux-gnu/libpthread.so.0 from remote target...
Reading /lib/x86_64-linux-gnu/libc.so.6 from remote target...
Reading /lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/libc-2.31.so from remote target...
Reading /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Reading /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Remote connection closed
...
Note the "Remote connection closed" message. That means GDBserver
exited abruptly.
I traced it down to the fact that GDB fetches the thread list from
GDBserver while the main thread of the process is still running. On
my main system where I wrote the testcase, I have not observed the
failure because it is slow enough that the thread stops before
GDBserver fetches the thread list in the problem scenario which I'll
describe below.
With some --remote-debug logging from GDBserver side, we see the last
packets before the connection closes:
...
getpkt ("vCont;c"); [no ack sent]
putpkt ("$OK#9a"); [noack mode]
getpkt ("Tp10f9a.10f9a"); [no ack sent]
putpkt ("$OK#9a"); [noack mode]
getpkt ("Hgp0.0"); [no ack sent]
putpkt ("$OK#9a"); [noack mode]
getpkt ("qXfer:threads:read::0,1000"); [no ack sent]
Note the vCont;c , which sets the program running, and then a
qXfer:threads:read packet at the end.
The problem happens when the thread list refresh (qXfer:threads:read)
is sent just while the main thread is running and it still hasn't
initialized its libpthread id internally. In that state, the main
thread's lwp will remain with the thread_known flag clear. See in
find_one_thread:
/* If the new thread ID is zero, a final thread ID will be available
later. Do not enable thread debugging yet. */
if (ti.ti_tid == 0)
return 0;
Now, back in server.cc, to handle the qXfer:threads:read, we reach
handle_qxfer_threads -> handle_qxfer_threads_proper, and the latter
then calls handle_qxfer_threads_worker for each known thread. In
handle_qxfer_threads_worker, we call target_thread_handle. This ends
up in thread_db_thread_handle, here:
if (!lwp->thread_known && !find_one_thread (thread->id))
return false;
Since the thread ID isn't known yet, we call find_one_thread. This
calls into libthread_db.so, which accesses memory. Because the
current thread is running, that fails and we throw an error, here:
/* Get information about this thread. */
err = thread_db->td_ta_map_lwp2thr_p (thread_db->thread_agent, lwpid, &th);
if (err != TD_OK)
error ("Cannot get thread handle for LWP %d: %s",
lwpid, thread_db_err_str (err));
The current design is that whenever GDB-facing packets/requests need
to accesses memory, server.cc is supposed to prepare the target for
the access. See gdb_read_memory / gdb_write_memory. This preparation
means pausing threads if in non-stop mode (someday we could lift this
requirement, but we will still need to pause to access registers or do
other related ptrace accesses like PTRACE_GET_THREAD_AREA). Note that
the multi-target.exp testcase forces "maint set target-non-stop on".
So the fix here is to prepare the target to access memory when
handling qXfer:threads:read too.
gdbserver/ChangeLog:
* inferiors.cc (switch_to_process): New, moved here from
thread-db.cc, and made extern.
* inferiors.h (switch_to_process): Declare.
* server.cc: Include "gdbsupport/scoped_restore.h".
(handle_qxfer_threads_proper): Now returns bool. Prepare to
access memory around target calls.
(handle_qxfer_threads): Handle errors.
* thread-db.cc (switch_to_process): Moved to inferiors.cc.
This page took 0.064058 seconds and 4 git commands to generate.