You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[LLVM] Emit dwarf data for changed-signature and new functions
Add a new pass EmitChangedFuncDebugInfo which will add dwarf for
additional functions whose signatures are changed during compiler
transformations.
The original intention is for bpf-based linux kernel tracing.
The function signature is available in vmlinux BTF generated
from pahole/dwarf. Such signature is generated from dwarf
at the source level. But this is not ideal since some function
may have signatures changed. If user still used the source
level signature, users may not get correct results and may
need some efforts to workaround the issue.
So we want to encode the true signature (not different
from the source one) in dwarf. With such additional information,
dwarf users can get these signature changed functions.
For example, pahole is able to process these signature
changed functions and encode them into vmlinux BTF properly.
History of multiple attempts
============================
Previously I have attempted a few tries ([1], [2] and [3]).
Initially I tried to modify debuginfo in passes like
ArgPromotion and DeadArgElim, but later on it is suggested
to have a central place to handle new signatures ([1]).
Later, I have another version of patch similar to this
one, but the recommendation is to modify debuginfo to
encode new signature within the same function,
either through inlinedAt or new signature overwriting
the old one. This seems working but it has some
side effect on lldb, some lldb output (e.g. back trace)
will be different from the previous one. The recommendation
is to avoid any behavior change for lldb ([2] and [3]).
So now, I came back to the solution discussed at the
end of [1]. Basically a special dwarf entry will be generated
to encode the new signature. The new signature will have
a reference to the old source-level signature.
So the tool can inspect dwarf to retrieve the related
info.
Examples and dwarf output
=========================
In below, a few examples will show how changed signatures
represented in dwarf:
Example 1
---------
Source:
$ cat test.c
struct t { int a; };
char *tar(struct t *a, struct t *d);
__attribute__((noinline)) static char * foo(struct t *a, int b, struct t *d)
{
return tar(a, d);
}
char *bar(struct t *a, struct t *d)
{
return foo(a, 1, d);
}
Compiled and dump dwarf with:
$ clang -O2 -c -g test.c
$ llvm-dwarfdump test.o
0x0000000c: DW_TAG_compile_unit
...
0x0000005c: DW_TAG_subprogram
DW_AT_low_pc (0x0000000000000010)
DW_AT_high_pc (0x0000000000000015)
DW_AT_frame_base (DW_OP_reg7 RSP)
DW_AT_call_all_calls (true)
DW_AT_name ("foo")
DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c")
DW_AT_decl_line (3)
DW_AT_prototyped (true)
DW_AT_calling_convention (DW_CC_nocall)
DW_AT_type (0x000000b1 "char *")
0x0000006c: DW_TAG_formal_parameter
DW_AT_location (DW_OP_reg5 RDI)
DW_AT_name ("a")
DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c")
DW_AT_decl_line (3)
DW_AT_type (0x000000ba "t *")
0x00000076: DW_TAG_formal_parameter
DW_AT_name ("b")
DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c")
DW_AT_decl_line (3)
DW_AT_type (0x000000ce "int")
0x0000007e: DW_TAG_formal_parameter
DW_AT_location (DW_OP_reg4 RSI)
DW_AT_name ("d")
DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c")
DW_AT_decl_line (3)
DW_AT_type (0x000000ba "t *")
0x00000088: DW_TAG_call_site
...
0x0000009d: NULL
...
0x000000d2: DW_TAG_inlined_subroutine
DW_AT_name ("foo")
DW_AT_type (0x000000b1 "char *")
DW_AT_artificial (true)
DW_AT_specification (0x0000005c "foo")
0x000000dc: DW_TAG_formal_parameter
DW_AT_name ("a")
DW_AT_type (0x000000ba "t *")
0x000000e2: DW_TAG_formal_parameter
DW_AT_name ("d")
DW_AT_type (0x000000ba "t *")
0x000000e8: NULL
In the above, the DISubprogram 'foo' has the original signature but
since parameter 'b' does not have DW_AT_location, it is clear that
parameter will not be used. The actual function signature is represented
in DW_TAG_inlined_subroutine.
For the above case, it looks like DW_TAG_inlined_subroutine is not
necessary. Let us try a few other examples below.
Example 2
---------
Source:
$ cat test.c
struct t { long a; long b;};
__attribute__((noinline)) static long foo(struct t arg) {
return arg.b * 5;
}
long bar(struct t arg) {
return foo(arg);
}
Compiled and dump dwarf with:
$ clang -O2 -c -g test.c
$ llvm-dwarfdump test.o
...
0x0000004e: DW_TAG_subprogram
DW_AT_low_pc (0x0000000000000010)
DW_AT_high_pc (0x0000000000000015)
DW_AT_frame_base (DW_OP_reg7 RSP)
DW_AT_call_all_calls (true)
DW_AT_name ("foo")
DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test.c")
DW_AT_decl_line (2)
DW_AT_prototyped (true)
DW_AT_calling_convention (DW_CC_nocall)
DW_AT_type (0x0000006d "long")
0x0000005e: DW_TAG_formal_parameter
DW_AT_location (DW_OP_piece 0x8, DW_OP_reg5 RDI, DW_OP_piece 0x8)
DW_AT_name ("arg")
DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test.c")
DW_AT_decl_line (2)
DW_AT_type (0x00000099 "t")
0x0000006c: NULL
...
0x00000088: DW_TAG_inlined_subroutine
DW_AT_name ("foo")
DW_AT_type (0x0000006d "long")
DW_AT_artificial (true)
DW_AT_specification (0x0000004e "foo")
0x00000092: DW_TAG_formal_parameter
DW_AT_name ("arg")
DW_AT_type (0x0000006d "long")
0x00000098: NULL
In the above case for function foo(), the original argument is 'struct t',
but the final actual argument is a 'long' type. DW_TAG_inlined_subroutine
can clearly represent the signature type instead of doing DW_AT_location
thing.
There is a problem in the above then, it is not clear what formal parameter
'arg' corresponds to the original parameter. If necessary, the compiler
could change 'arg' to e.g. 'arg_offset_8' to indicate it is 8 byte offset from
the original struct.
Example 3
---------
Source:
$ cat test2.c
struct t { long a; long b; long c;};
__attribute__((noinline)) long foo(struct t arg) {
return arg.a * arg.c;
}
long bar(struct t arg) {
return foo(arg);
}
Compiled and dump dwarf with:
$ clang -O2 -c -g test2.c
$ llvm-dwarfdump test2.o
...
0x0000003e: DW_TAG_subprogram
DW_AT_low_pc (0x0000000000000010)
DW_AT_high_pc (0x0000000000000015)
DW_AT_frame_base (DW_OP_reg7 RSP)
DW_AT_call_all_calls (true)
DW_AT_name ("bar")
DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test2.c")
DW_AT_decl_line (5)
DW_AT_prototyped (true)
DW_AT_type (0x0000005f "long")
DW_AT_external (true)
0x0000004d: DW_TAG_formal_parameter
DW_AT_location (DW_OP_fbreg +8)
DW_AT_name ("arg")
DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test2.c")
DW_AT_decl_line (5)
DW_AT_type (0x00000079 "t")
0x00000058: DW_TAG_call_site
DW_AT_call_origin (0x00000023 "foo")
DW_AT_call_tail_call (true)
DW_AT_call_pc (0x0000000000000010)
0x0000005e: NULL
...
0x00000063: DW_TAG_inlined_subroutine
DW_AT_name ("foo")
DW_AT_type (0x0000005f "long")
DW_AT_artificial (true)
DW_AT_specification (0x00000023 "foo")
0x0000006d: DW_TAG_formal_parameter
DW_AT_name ("arg")
DW_AT_type (0x00000074 "t *")
0x00000073: NULL
In the above example, from DW_TAG_subprogram, it is not clear what kind
of type the parameter should be. But DW_TAG_inlined_subroutine can
clearly show what the type should be. Again, the name can be changed
e.g. 'arg_ptr' if desired.
Example 4
---------
Source:
$ cat test.c
__attribute__((noinline)) static int callee(const int *p) { return *p + 42; }
int caller(void) {
int x = 100;
return callee(&x);
}
Compiled and dump dwarf with:
$ clang -O3 -c -g test2.c
$ llvm-dwarfdump test2.o
...
0x0000004a: DW_TAG_subprogram
DW_AT_low_pc (0x0000000000000010)
DW_AT_high_pc (0x0000000000000014)
DW_AT_frame_base (DW_OP_reg7 RSP)
DW_AT_call_all_calls (true)
DW_AT_name ("callee")
DW_AT_decl_file ("/home/yhs/tests/sig-change/prom/test.c")
DW_AT_decl_line (1)
DW_AT_prototyped (true)
DW_AT_calling_convention (DW_CC_nocall)
DW_AT_type (0x00000063 "int")
0x0000005a: DW_TAG_formal_parameter
DW_AT_name ("p")
DW_AT_decl_file ("/home/yhs/tests/sig-change/prom/test.c")
DW_AT_decl_line (1)
DW_AT_type (0x00000078 "const int *")
0x00000062: NULL
...
0x00000067: DW_TAG_inlined_subroutine
DW_AT_name ("callee")
DW_AT_type (0x00000063 "int")
DW_AT_artificial (true)
DW_AT_specification (0x0000004a "callee")
0x00000071: DW_TAG_formal_parameter
DW_AT_name ("__0")
DW_AT_type (0x00000063 "int")
0x00000077: NULL
In the above, the function
static int callee(const int *p) { return *p + 42; }
is transformed to
static int callee(int p) { return p + 42; }
But the new signature is not reflected in DW_TAG_subprogram.
The DW_TAG_inlined_subroutine can precisely capture the
signature. Note that the parameter name is "__0" and "0" means
the first argument. The reason is due to the following IR:
define internal ... i32 @callee(i32 %0) unnamed_addr #1 !dbg !23 {
#dbg_value(ptr poison, !29, !DIExpression(), !30)
%2 = add nsw i32 %0, 42, !dbg !31
ret i32 %2, !dbg !32
}
...
!29 = !DILocalVariable(name: "p", arg: 1, scope: !23, file: !1, line: 1, type: !26)
The reason is due to 'ptr poison' as 'ptr poison' mean the debug
value should not be used any more. This is also the reason that
the above DW_TAG_subprogram does not have location information.
DW_TAG_inlined_subroutine can provide correct signature though.
If we compile like below:
clang -O3 -c -g test.c -fno-discard-value-names
The function argument name will be preserved
... i32 @callee(i32 %p.0.val) ...
and in such cases,
the DW_TAG_inlined_subroutine looks like below:
0x00000067: DW_TAG_inlined_subroutine
DW_AT_name ("callee")
DW_AT_type (0x00000063 "int")
DW_AT_artificial (true)
DW_AT_specification (0x0000004a "callee")
0x00000071: DW_TAG_formal_parameter
DW_AT_name ("p__0__val")
DW_AT_type (0x00000063 "int")
0x00000077: NULL
Note that the original argument name replaces '.' with "__"
so argument name has proper C standard.
Based a run on linux kernel, the names like "__<arg_index>"
roughly 2% of total signature changed functions, so we probably
okay for now.
Non-LTO vs. LTO
---------------
For thin-lto mode, we often see kernel symbols like
p9_req_cache.llvm.13472271643223911678
Even if this symbol has identical source level signature with p9_req_cache,
a special DW_TAG_inlined_subroutine will be generated with
name 'p9_req_cache.llvm.13472271643223911678'.
With this, some tool (e.g., pahole) may generate a BTF entry
for this name which could be used for fentry/fexit tracing.
But if a symbol with "<foo>.llvm.<hash>" has different signatures
than the source level "<foo>", then a special DW_TAG_inlined_subroutine
will be generated like below:
0x10f0793f: DW_TAG_inlined_subroutine
DW_AT_name ("flow_offload_fill_route")
DW_AT_linkage_name ("flow_offload_fill_route.llvm.14555965973926298225")
DW_AT_artificial (true)
DW_AT_specification (0x10ee9e54 "flow_offload_fill_route")
0x10f07949: DW_TAG_formal_parameter
DW_AT_name ("flow")
DW_AT_type (0x10ee837a "flow_offload *")
0x10f07951: DW_TAG_formal_parameter
DW_AT_name ("route")
DW_AT_type (0x10eea4ef "nf_flow_route *")
0x10f07959: DW_TAG_formal_parameter
DW_AT_name ("dir")
DW_AT_type (0x10ecef15 "int")
0x10f07961: NULL
In the above, function "flow_offload_fill_route" has return type
"int" at source level, but optimization eventually made the return
type as "void". The tools like pahole may choice to generate
two entries with DW_AT_name and DW_AT_linkage_name for vmlinux BTF.
Note that it is possible one source symbol may have multiple linkage
name's due to potentially (more than one) cloning in llvm. In such
cases, multiple DW_TAG_inlined_subroutine instances might be possible.
Some restrictions
=================
There are some restrictions in the current implementation:
- Only C language is supported
- BPF target is excluded as one of main goals for this pull request
is to generate proper vmlinux BTF for arch's like x86_64/arm64 etc.
- Function must not be a intrinsic, decl only, return value size more
than arch register size and func with variable arguments.
- For arguments, only int/ptr types are supported.
- Some union type arguments (e.g., 8B < union_size <= 16B) may
have DIType issue so some function may be skipped.
Some statistics with linux kernel
=================================
I have tested this patch set by building latest bpf-next linux kernel.
For no-lto case:
65341 original number of functions
1054 signature changed functions with this patch
For thin-lto case:
65595 original number of functions
3150 signature changed functions with this patch
Next step
=========
With this llvm change, we will be able to do some work in pahole and libbpf.
For pahole, currently we will see the warning:
die__process_unit: DW_TAG_inlined_subroutine (0x1d) @ <0xf2db986> not handled in a c11 CU!
Basically these DW_TAG_inlined_subroutine are not inside the DISubprogram.
[1] #127855
[2] #157349
[3] https://discourse.llvm.org/t/rfc-identify-func-signature-change-in-llvm-compiled-kernel-image/82609
0 commit comments