Skip to content

Commit 723113d

Browse files
author
Yonghong Song
committed
[LLVM] Emit dwarf data for changed-signature and new functions
Add a new pass EmitChangedFuncDebugInfo which will add dwarf for additional functions whose signatures are changed during compiler transformations. The original intention is for bpf-based linux kernel tracing. The function signature is available in vmlinux BTF generated from pahole/dwarf. Such signature is generated from dwarf at the source level. But this is not ideal since some function may have signatures changed. If user still used the source level signature, users may not get correct results and may need some efforts to workaround the issue. So we want to encode the true signature (not different from the source one) in dwarf. With such additional information, dwarf users can get these signature changed functions. For example, pahole is able to process these signature changed functions and encode them into vmlinux BTF properly. History of multiple attempts ============================ Previously I have attempted a few tries ([1], [2] and [3]). Initially I tried to modify debuginfo in passes like ArgPromotion and DeadArgElim, but later on it is suggested to have a central place to handle new signatures ([1]). Later, I have another version of patch similar to this one, but the recommendation is to modify debuginfo to encode new signature within the same function, either through inlinedAt or new signature overwriting the old one. This seems working but it has some side effect on lldb, some lldb output (e.g. back trace) will be different from the previous one. The recommendation is to avoid any behavior change for lldb ([2] and [3]). So now, I came back to the solution discussed at the end of [1]. Basically a special dwarf entry will be generated to encode the new signature. The new signature will have a reference to the old source-level signature. So the tool can inspect dwarf to retrieve the related info. Examples and dwarf output ========================= In below, a few examples will show how changed signatures represented in dwarf: Example 1 --------- Source: $ cat test.c struct t { int a; }; char *tar(struct t *a, struct t *d); __attribute__((noinline)) static char * foo(struct t *a, int b, struct t *d) { return tar(a, d); } char *bar(struct t *a, struct t *d) { return foo(a, 1, d); } Compiled and dump dwarf with: $ clang -O2 -c -g test.c $ llvm-dwarfdump test.o 0x0000000c: DW_TAG_compile_unit ... 0x0000005c: DW_TAG_subprogram DW_AT_low_pc (0x0000000000000010) DW_AT_high_pc (0x0000000000000015) DW_AT_frame_base (DW_OP_reg7 RSP) DW_AT_call_all_calls (true) DW_AT_name ("foo") DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c") DW_AT_decl_line (3) DW_AT_prototyped (true) DW_AT_calling_convention (DW_CC_nocall) DW_AT_type (0x000000b1 "char *") 0x0000006c: DW_TAG_formal_parameter DW_AT_location (DW_OP_reg5 RDI) DW_AT_name ("a") DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c") DW_AT_decl_line (3) DW_AT_type (0x000000ba "t *") 0x00000076: DW_TAG_formal_parameter DW_AT_name ("b") DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c") DW_AT_decl_line (3) DW_AT_type (0x000000ce "int") 0x0000007e: DW_TAG_formal_parameter DW_AT_location (DW_OP_reg4 RSI) DW_AT_name ("d") DW_AT_decl_file ("/home/yhs/tests/sig-change/deadarg/test.c") DW_AT_decl_line (3) DW_AT_type (0x000000ba "t *") 0x00000088: DW_TAG_call_site ... 0x0000009d: NULL ... 0x000000d2: DW_TAG_inlined_subroutine DW_AT_name ("foo") DW_AT_type (0x000000b1 "char *") DW_AT_artificial (true) DW_AT_specification (0x0000005c "foo") 0x000000dc: DW_TAG_formal_parameter DW_AT_name ("a") DW_AT_type (0x000000ba "t *") 0x000000e2: DW_TAG_formal_parameter DW_AT_name ("d") DW_AT_type (0x000000ba "t *") 0x000000e8: NULL In the above, the DISubprogram 'foo' has the original signature but since parameter 'b' does not have DW_AT_location, it is clear that parameter will not be used. The actual function signature is represented in DW_TAG_inlined_subroutine. For the above case, it looks like DW_TAG_inlined_subroutine is not necessary. Let us try a few other examples below. Example 2 --------- Source: $ cat test.c struct t { long a; long b;}; __attribute__((noinline)) static long foo(struct t arg) { return arg.b * 5; } long bar(struct t arg) { return foo(arg); } Compiled and dump dwarf with: $ clang -O2 -c -g test.c $ llvm-dwarfdump test.o ... 0x0000004e: DW_TAG_subprogram DW_AT_low_pc (0x0000000000000010) DW_AT_high_pc (0x0000000000000015) DW_AT_frame_base (DW_OP_reg7 RSP) DW_AT_call_all_calls (true) DW_AT_name ("foo") DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test.c") DW_AT_decl_line (2) DW_AT_prototyped (true) DW_AT_calling_convention (DW_CC_nocall) DW_AT_type (0x0000006d "long") 0x0000005e: DW_TAG_formal_parameter DW_AT_location (DW_OP_piece 0x8, DW_OP_reg5 RDI, DW_OP_piece 0x8) DW_AT_name ("arg") DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test.c") DW_AT_decl_line (2) DW_AT_type (0x00000099 "t") 0x0000006c: NULL ... 0x00000088: DW_TAG_inlined_subroutine DW_AT_name ("foo") DW_AT_type (0x0000006d "long") DW_AT_artificial (true) DW_AT_specification (0x0000004e "foo") 0x00000092: DW_TAG_formal_parameter DW_AT_name ("arg") DW_AT_type (0x0000006d "long") 0x00000098: NULL In the above case for function foo(), the original argument is 'struct t', but the final actual argument is a 'long' type. DW_TAG_inlined_subroutine can clearly represent the signature type instead of doing DW_AT_location thing. There is a problem in the above then, it is not clear what formal parameter 'arg' corresponds to the original parameter. If necessary, the compiler could change 'arg' to e.g. 'arg_offset_8' to indicate it is 8 byte offset from the original struct. Example 3 --------- Source: $ cat test2.c struct t { long a; long b; long c;}; __attribute__((noinline)) long foo(struct t arg) { return arg.a * arg.c; } long bar(struct t arg) { return foo(arg); } Compiled and dump dwarf with: $ clang -O2 -c -g test2.c $ llvm-dwarfdump test2.o ... 0x0000003e: DW_TAG_subprogram DW_AT_low_pc (0x0000000000000010) DW_AT_high_pc (0x0000000000000015) DW_AT_frame_base (DW_OP_reg7 RSP) DW_AT_call_all_calls (true) DW_AT_name ("bar") DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test2.c") DW_AT_decl_line (5) DW_AT_prototyped (true) DW_AT_type (0x0000005f "long") DW_AT_external (true) 0x0000004d: DW_TAG_formal_parameter DW_AT_location (DW_OP_fbreg +8) DW_AT_name ("arg") DW_AT_decl_file ("/home/yhs/tests/sig-change/struct/test2.c") DW_AT_decl_line (5) DW_AT_type (0x00000079 "t") 0x00000058: DW_TAG_call_site DW_AT_call_origin (0x00000023 "foo") DW_AT_call_tail_call (true) DW_AT_call_pc (0x0000000000000010) 0x0000005e: NULL ... 0x00000063: DW_TAG_inlined_subroutine DW_AT_name ("foo") DW_AT_type (0x0000005f "long") DW_AT_artificial (true) DW_AT_specification (0x00000023 "foo") 0x0000006d: DW_TAG_formal_parameter DW_AT_name ("arg") DW_AT_type (0x00000074 "t *") 0x00000073: NULL In the above example, from DW_TAG_subprogram, it is not clear what kind of type the parameter should be. But DW_TAG_inlined_subroutine can clearly show what the type should be. Again, the name can be changed e.g. 'arg_ptr' if desired. Example 4 --------- Source: $ cat test.c __attribute__((noinline)) static int callee(const int *p) { return *p + 42; } int caller(void) { int x = 100; return callee(&x); } Compiled and dump dwarf with: $ clang -O3 -c -g test2.c $ llvm-dwarfdump test2.o ... 0x0000004a: DW_TAG_subprogram DW_AT_low_pc (0x0000000000000010) DW_AT_high_pc (0x0000000000000014) DW_AT_frame_base (DW_OP_reg7 RSP) DW_AT_call_all_calls (true) DW_AT_name ("callee") DW_AT_decl_file ("/home/yhs/tests/sig-change/prom/test.c") DW_AT_decl_line (1) DW_AT_prototyped (true) DW_AT_calling_convention (DW_CC_nocall) DW_AT_type (0x00000063 "int") 0x0000005a: DW_TAG_formal_parameter DW_AT_name ("p") DW_AT_decl_file ("/home/yhs/tests/sig-change/prom/test.c") DW_AT_decl_line (1) DW_AT_type (0x00000078 "const int *") 0x00000062: NULL ... 0x00000067: DW_TAG_inlined_subroutine DW_AT_name ("callee") DW_AT_type (0x00000063 "int") DW_AT_artificial (true) DW_AT_specification (0x0000004a "callee") 0x00000071: DW_TAG_formal_parameter DW_AT_name ("__0") DW_AT_type (0x00000063 "int") 0x00000077: NULL In the above, the function static int callee(const int *p) { return *p + 42; } is transformed to static int callee(int p) { return p + 42; } But the new signature is not reflected in DW_TAG_subprogram. The DW_TAG_inlined_subroutine can precisely capture the signature. Note that the parameter name is "__0" and "0" means the first argument. The reason is due to the following IR: define internal ... i32 @callee(i32 %0) unnamed_addr #1 !dbg !23 { #dbg_value(ptr poison, !29, !DIExpression(), !30) %2 = add nsw i32 %0, 42, !dbg !31 ret i32 %2, !dbg !32 } ... !29 = !DILocalVariable(name: "p", arg: 1, scope: !23, file: !1, line: 1, type: !26) The reason is due to 'ptr poison' as 'ptr poison' mean the debug value should not be used any more. This is also the reason that the above DW_TAG_subprogram does not have location information. DW_TAG_inlined_subroutine can provide correct signature though. If we compile like below: clang -O3 -c -g test.c -fno-discard-value-names The function argument name will be preserved ... i32 @callee(i32 %p.0.val) ... and in such cases, the DW_TAG_inlined_subroutine looks like below: 0x00000067: DW_TAG_inlined_subroutine DW_AT_name ("callee") DW_AT_type (0x00000063 "int") DW_AT_artificial (true) DW_AT_specification (0x0000004a "callee") 0x00000071: DW_TAG_formal_parameter DW_AT_name ("p__0__val") DW_AT_type (0x00000063 "int") 0x00000077: NULL Note that the original argument name replaces '.' with "__" so argument name has proper C standard. Based a run on linux kernel, the names like "__<arg_index>" roughly 2% of total signature changed functions, so we probably okay for now. Non-LTO vs. LTO --------------- For thin-lto mode, we often see kernel symbols like p9_req_cache.llvm.13472271643223911678 Even if this symbol has identical source level signature with p9_req_cache, a special DW_TAG_inlined_subroutine will be generated with name 'p9_req_cache.llvm.13472271643223911678'. With this, some tool (e.g., pahole) may generate a BTF entry for this name which could be used for fentry/fexit tracing. But if a symbol with "<foo>.llvm.<hash>" has different signatures than the source level "<foo>", then a special DW_TAG_inlined_subroutine will be generated like below: 0x10f0793f: DW_TAG_inlined_subroutine DW_AT_name ("flow_offload_fill_route") DW_AT_linkage_name ("flow_offload_fill_route.llvm.14555965973926298225") DW_AT_artificial (true) DW_AT_specification (0x10ee9e54 "flow_offload_fill_route") 0x10f07949: DW_TAG_formal_parameter DW_AT_name ("flow") DW_AT_type (0x10ee837a "flow_offload *") 0x10f07951: DW_TAG_formal_parameter DW_AT_name ("route") DW_AT_type (0x10eea4ef "nf_flow_route *") 0x10f07959: DW_TAG_formal_parameter DW_AT_name ("dir") DW_AT_type (0x10ecef15 "int") 0x10f07961: NULL In the above, function "flow_offload_fill_route" has return type "int" at source level, but optimization eventually made the return type as "void". The tools like pahole may choice to generate two entries with DW_AT_name and DW_AT_linkage_name for vmlinux BTF. Note that it is possible one source symbol may have multiple linkage name's due to potentially (more than one) cloning in llvm. In such cases, multiple DW_TAG_inlined_subroutine instances might be possible. Some restrictions ================= There are some restrictions in the current implementation: - Only C language is supported - BPF target is excluded as one of main goals for this pull request is to generate proper vmlinux BTF for arch's like x86_64/arm64 etc. - Function must not be a intrinsic, decl only, return value size more than arch register size and func with variable arguments. - For arguments, only int/ptr types are supported. - Some union type arguments (e.g., 8B < union_size <= 16B) may have DIType issue so some function may be skipped. Some statistics with linux kernel ================================= I have tested this patch set by building latest bpf-next linux kernel. For no-lto case: 65341 original number of functions 1054 signature changed functions with this patch For thin-lto case: 65595 original number of functions 3150 signature changed functions with this patch Next step ========= With this llvm change, we will be able to do some work in pahole and libbpf. For pahole, currently we will see the warning: die__process_unit: DW_TAG_inlined_subroutine (0x1d) @ <0xf2db986> not handled in a c11 CU! Basically these DW_TAG_inlined_subroutine are not inside the DISubprogram. [1] #127855 [2] #157349 [3] https://discourse.llvm.org/t/rfc-identify-func-signature-change-in-llvm-compiled-kernel-image/82609
1 parent c169d11 commit 723113d

20 files changed

+1534
-2
lines changed
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
//===- EmitChangedFuncDebugInfo.h - Emit Additional Debug Info -*- C++ --*-===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
//
9+
/// \file
10+
/// Emit debug info for changed or new funcs.
11+
//===----------------------------------------------------------------------===//
12+
13+
#ifndef LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H
14+
#define LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H
15+
16+
#include "llvm/IR/PassManager.h"
17+
18+
namespace llvm {
19+
20+
class Module;
21+
22+
// Pass that emits late dwarf.
23+
class EmitChangedFuncDebugInfoPass
24+
: public PassInfoMixin<EmitChangedFuncDebugInfoPass> {
25+
public:
26+
EmitChangedFuncDebugInfoPass() = default;
27+
28+
PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
29+
};
30+
31+
} // end namespace llvm
32+
33+
#endif // LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1280,11 +1280,77 @@ void DwarfDebug::finishSubprogramDefinitions() {
12801280
}
12811281
}
12821282

1283+
void DwarfDebug::addChangedSubprograms() {
1284+
// Generate additional dwarf for functions with signature changed.
1285+
DICompileUnit *ExtraCU = nullptr;
1286+
for (DICompileUnit *CUNode : MMI->getModule()->debug_compile_units()) {
1287+
if (CUNode->getFile()->getFilename() == "<changed_signatures>") {
1288+
ExtraCU = CUNode;
1289+
break;
1290+
}
1291+
}
1292+
if (!ExtraCU)
1293+
return;
1294+
1295+
llvm::DebugInfoFinder DIF;
1296+
DIF.processModule(*MMI->getModule());
1297+
for (auto *ExtraSP : DIF.subprograms()) {
1298+
if (ExtraSP->getUnit() != ExtraCU)
1299+
continue;
1300+
1301+
DISubprogram *SP = cast<DISubprogram>(ExtraSP->getScope());
1302+
DwarfCompileUnit &Cu = getOrCreateDwarfCompileUnit(SP->getUnit());
1303+
DIE *ScopeDIE =
1304+
DIE::get(DIEValueAllocator, dwarf::DW_TAG_inlined_subroutine);
1305+
Cu.getUnitDie().addChild(ScopeDIE);
1306+
1307+
Cu.addString(*ScopeDIE, dwarf::DW_AT_name, ExtraSP->getName());
1308+
if (!ExtraSP->getLinkageName().empty())
1309+
Cu.addString(*ScopeDIE, dwarf::DW_AT_linkage_name,
1310+
ExtraSP->getLinkageName());
1311+
1312+
DITypeRefArray Args = ExtraSP->getType()->getTypeArray();
1313+
1314+
if (Args[0])
1315+
Cu.addType(*ScopeDIE, Args[0]);
1316+
1317+
if (ExtraSP->getType()->getCC() == llvm::dwarf::DW_CC_nocall) {
1318+
Cu.addUInt(*ScopeDIE, dwarf::DW_AT_calling_convention,
1319+
dwarf::DW_FORM_data1, llvm::dwarf::DW_CC_nocall);
1320+
}
1321+
1322+
Cu.addFlag(*ScopeDIE, dwarf::DW_AT_artificial);
1323+
1324+
// dereference the DIE* for DIEEntry
1325+
DIE *OriginDIE = Cu.getOrCreateSubprogramDIE(SP, nullptr);
1326+
Cu.addDIEEntry(*ScopeDIE, dwarf::DW_AT_specification, DIEEntry(*OriginDIE));
1327+
1328+
SmallVector<const DILocalVariable *> ArgVars(Args.size());
1329+
for (const DINode *DN : ExtraSP->getRetainedNodes()) {
1330+
if (const auto *DV = dyn_cast<DILocalVariable>(DN)) {
1331+
uint32_t Arg = DV->getArg();
1332+
if (Arg)
1333+
ArgVars[Arg - 1] = DV;
1334+
}
1335+
}
1336+
1337+
for (unsigned i = 1, N = Args.size(); i < N; ++i) {
1338+
const DIType *Ty = Args[i];
1339+
DIE &Arg = Cu.createAndAddDIE(dwarf::DW_TAG_formal_parameter, *ScopeDIE);
1340+
const DILocalVariable *DV = ArgVars[i - 1];
1341+
Cu.addString(Arg, dwarf::DW_AT_name, DV->getName());
1342+
Cu.addType(Arg, Ty);
1343+
}
1344+
}
1345+
}
1346+
12831347
void DwarfDebug::finalizeModuleInfo() {
12841348
const TargetLoweringObjectFile &TLOF = Asm->getObjFileLowering();
12851349

12861350
finishSubprogramDefinitions();
12871351

1352+
addChangedSubprograms();
1353+
12881354
finishEntityDefinitions();
12891355

12901356
bool HasEmittedSplitCU = false;

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -565,6 +565,8 @@ class DwarfDebug : public DebugHandlerBase {
565565

566566
void finishSubprogramDefinitions();
567567

568+
void addChangedSubprograms();
569+
568570
/// Finish off debug information after all functions have been
569571
/// processed.
570572
void finalizeModuleInfo();

llvm/lib/Passes/PassBuilder.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -351,6 +351,7 @@
351351
#include "llvm/Transforms/Utils/DXILUpgrade.h"
352352
#include "llvm/Transforms/Utils/Debugify.h"
353353
#include "llvm/Transforms/Utils/DeclareRuntimeLibcalls.h"
354+
#include "llvm/Transforms/Utils/EmitChangedFuncDebugInfo.h"
354355
#include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
355356
#include "llvm/Transforms/Utils/FixIrreducible.h"
356357
#include "llvm/Transforms/Utils/HelloWorld.h"

llvm/lib/Passes/PassBuilderPipelines.cpp

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,7 @@
135135
#include "llvm/Transforms/Utils/AssumeBundleBuilder.h"
136136
#include "llvm/Transforms/Utils/CanonicalizeAliases.h"
137137
#include "llvm/Transforms/Utils/CountVisits.h"
138+
#include "llvm/Transforms/Utils/EmitChangedFuncDebugInfo.h"
138139
#include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
139140
#include "llvm/Transforms/Utils/ExtraPassManager.h"
140141
#include "llvm/Transforms/Utils/InjectTLIMappings.h"
@@ -1637,9 +1638,12 @@ PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
16371638
if (PTO.CallGraphProfile && !LTOPreLink)
16381639
MPM.addPass(CGProfilePass(isLTOPostLink(LTOPhase)));
16391640

1640-
// RelLookupTableConverterPass runs later in LTO post-link pipeline.
1641-
if (!LTOPreLink)
1641+
// RelLookupTableConverterPass and EmitChangedFuncDebugInfoPass run later in
1642+
// LTO post-link pipeline.
1643+
if (!LTOPreLink) {
16421644
MPM.addPass(RelLookupTableConverterPass());
1645+
MPM.addPass(EmitChangedFuncDebugInfoPass());
1646+
}
16431647

16441648
return MPM;
16451649
}

llvm/lib/Passes/PassRegistry.def

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ MODULE_PASS("dfsan", DataFlowSanitizerPass())
7575
MODULE_PASS("dot-callgraph", CallGraphDOTPrinterPass())
7676
MODULE_PASS("dxil-upgrade", DXILUpgradePass())
7777
MODULE_PASS("elim-avail-extern", EliminateAvailableExternallyPass())
78+
MODULE_PASS("emit-changed-func-debuginfo", EmitChangedFuncDebugInfoPass())
7879
MODULE_PASS("extract-blocks", BlockExtractorPass({}, false))
7980
MODULE_PASS("expand-variadics",
8081
ExpandVariadicsPass(ExpandVariadicsMode::Disable))

llvm/lib/Transforms/Utils/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ add_llvm_component_library(LLVMTransformUtils
2323
DebugSSAUpdater.cpp
2424
DeclareRuntimeLibcalls.cpp
2525
DemoteRegToStack.cpp
26+
EmitChangedFuncDebugInfo.cpp
2627
DXILUpgrade.cpp
2728
EntryExitInstrumenter.cpp
2829
EscapeEnumerator.cpp

0 commit comments

Comments
 (0)