[BOLT][AArch64] unable to run BOLT on libffi

[geofft]

We originally ran into this building CPython for python-build-standalone, but here's a minimal reproducer:

ubuntu@ubuntu:~/libffi-3.5.1$ ./configure CFLAGS=-Wl,-emit-relocs
ubuntu@ubuntu:~/libffi-3.5.1$ make -j8
ubuntu@ubuntu:~/libffi-3.5.1$ llvm-bolt-21 aarch64-unknown-linux-gnu/.libs/libffi.so.8.2.0 -o=foo
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: <unknown>
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x200000, offset 0x200000
BOLT-INFO: enabling relocation mode
BOLT-WARNING: function ffi_go_closure_SYSV/1 has an object detected in a padding region at address 0x18048
BOLT-WARNING: function __clear_cache/1 has an object detected in a padding region at address 0x18068
BOLT-WARNING: 1 collisions detected while hashing binary objects. Use -v=1 to see the list.
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 0 out of 134 functions in the binary (0.0%) have non-empty execution profile
BOLT-INFO: removed 17 empty blocks
BOLT-ERROR: cannot relax ADR in non-simple function trampoline_code_table/1

This is with https://github.com/libffi/libffi/releases/download/v3.5.1/libffi-3.5.1.tar.gz, Ubuntu Plucky, and the the current version from apt.llvm.org (bolt-21=1:21~++20250630095742+59a7185dd9d6-1~exp1~20250630215911.150).

I think this is because trampoline_code_table is defined as hand-written assembly that does an adr and has no nops for BOLT to work with.

A naive attempt at throwing some extra nops into that function gave a worse failure, which I guess makes sense because it's loading some PC-relative addresses and I surely got those addresses wrong. But also it would be cool to be able to run BOLT without requiring changes in third-party code.

FWIW I think I would be happy leaving this un-optimized, if that's an option, but the adr relaxation pass looks unconditional and it seems to error out instead of skipping optimizing this one function. I don't totally understand whether this pass really needs to be mandatory.

[llvmbot]

@llvm/issue-subscribers-bolt

Author: Geoffrey Thomas (geofft)

We originally ran into this building CPython for python-build-standalone, but here's a minimal reproducer:

ubuntu@<!-- -->ubuntu:~/libffi-3.5.1$ ./configure CFLAGS=-Wl,-emit-relocs
ubuntu@<!-- -->ubuntu:~/libffi-3.5.1$ make -j8
ubuntu@<!-- -->ubuntu:~/libffi-3.5.1$ llvm-bolt-21 aarch64-unknown-linux-gnu/.libs/libffi.so.8.2.0 -o=foo
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: &lt;unknown&gt;
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x200000, offset 0x200000
BOLT-INFO: enabling relocation mode
BOLT-WARNING: function ffi_go_closure_SYSV/1 has an object detected in a padding region at address 0x18048
BOLT-WARNING: function __clear_cache/1 has an object detected in a padding region at address 0x18068
BOLT-WARNING: 1 collisions detected while hashing binary objects. Use -v=1 to see the list.
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 0 out of 134 functions in the binary (0.0%) have non-empty execution profile
BOLT-INFO: removed 17 empty blocks
BOLT-ERROR: cannot relax ADR in non-simple function trampoline_code_table/1

This is with https://github.com/libffi/libffi/releases/download/v3.5.1/libffi-3.5.1.tar.gz, Ubuntu Plucky, and the the current version from apt.llvm.org (bolt-21=1:21~++20250630095742+59a7185dd9d6-1exp120250630215911.150).

I think this is because trampoline_code_table is defined as hand-written assembly that does an adr and has no nops for BOLT to work with.

A naive attempt at throwing some extra nops into that function gave a worse failure, which I guess makes sense because it's loading some PC-relative addresses and I surely got those addresses wrong. But also it would be cool to be able to run BOLT without requiring changes in third-party code.

FWIW I think I would be happy leaving this un-optimized, if that's an option, but the adr relaxation pass looks unconditional and it seems to error out instead of skipping optimizing this one function. I don't totally understand whether this pass really needs to be mandatory.

[aaupov]

If I'm reading BOLT package version correctly, it's based on a recent trunk rev 59a7185, meaning all most recent fixes are present.

BOLT-ERROR: cannot relax ADR in non-simple function trampoline_code_table/1

@maksfb - what's the recommendation in this case? I suppose lite mode and compact-code-model should be enabled.

FWIW I think I would be happy leaving this un-optimized, if that's an option, but the adr relaxation pass looks unconditional and it seems to error out instead of skipping optimizing this one function. I don't totally understand whether this pass really needs to be mandatory.

Relaxing ADRs enables the hot-cold splitting, function reordering, and only processing profiled functions. Otherwise BOLT is limited in what it can do, hence this pass is mandatory on AArch64 in relocation mode. Disabling relocation mode (-relocs=0) may help work around it, but it comes with a performance penalty.

[geofft]

If I'm reading BOLT package version correctly, it's based on a recent trunk rev 59a7185, meaning all most recent fixes are present.

Yup, the minimized reproducer uses the nightly builds at apt.llvm.org. I forgot to mention, the prod issue happened with the released version of LLVM 20.1.4 (built via https://github.com/indygreg/toolchain-tools/tree/main/toolchain-bootstrap), which gave the same error on trying to BOLT the entire Python binary with a statically-linked libffi (but with a slightly older error message).

Is there something we can do to modify the inline assembly to be written in a way that BOLT is happy with? We're already carrying a patch to libffi (though, ideally, I would like something I can upstream into libffi).

Alternatively can I get BOLT to skip optimizing this one function and optimize everything else? Or is that not a coherent thing to ask?

[maksfb]

I'm going to take a closer look later. The first option is to add the source of ADR and the destination to --skip-funcs list. If the destination is data, then just the source. I plan to make this happen automatically and it's good to have a test case.

[yavtuk]

@geofft can you patch ffi ? if so you can add the following

file: src/aarch64/sysv.S

@@ -150,8 +150,9 @@ CNAME(ffi_call_SYSV):
        ldp     x29, x30, [x29]

        /* Save the return value as directed.  */
-       adr     x5, 0f
        and     w4, w4, #AARCH64_RET_MASK
+       adrp    x5, 0f
+       add     x5, x5, 0f
        add     x5, x5, x4, lsl #4
        br      x5

@@ -379,7 +380,8 @@ L(do_closure):
        bl      CNAME(ffi_closure_SYSV_inner)

        /* Load the return value as directed.  */
-       adr     x1, 0f
+       adrp    x1, 0f
+       add     x1, x1, 0f
        and     w0, w0, #AARCH64_RET_MASK
        add     x1, x1, x0, lsl #4
        add     x3, sp, #16+CALL_CONTEXT_SIZE

        .align  AARCH64_TRAMP_MAP_SHIFT
 CNAME(trampoline_code_table):
+       .set    table_offset,0x0
        .rept   AARCH64_TRAMP_MAP_SIZE / AARCH64_TRAMP_SIZE
        sub     sp, sp, #16             /* Make space on the stack */
        str     x17, [sp]               /* Save x17 on stack */
-       adr     x17, #16376             /* Get data address */
-       ldr     x17, [x17]              /* Copy data into x17 */
+       adrp    x17, ffi_go_closure_SYSV_V
+       ldr     x17, [x17, #table_offset & 0xfff]               /* Copy data into x17 */
        str     x17, [sp, #8]           /* Save data on stack */
-       adr     x17, #16372             /* Get code address */
-       ldr     x17, [x17]              /* Load code address into x17 */
+       adrp    x17, ffi_go_closure_SYSV_V
+       ldr     x17, [x17, #table_offset+8 & 0xfff]             /* Copy data into x17 */
        br      x17                     /* Jump to code */
+       .set    table_offset, table_offset + AARCH64_TRAMP_SIZE
        .endr

[maksfb]

@yavtuk is the libffi patch above specific to golang?

[maksfb]

@geofft, you can also try to add --lite option in addition to --skip-funcs=....

[yavtuk]

@yavtuk is the libffi patch above specific to golang?

No, I just looked closer to the libffi library and found the asm file related to aarch64, inside the one there are only 3 adr instructions (one of them for Mach specific function). In my opinion, until we don't support JT with 100% probability it's better to fix adr by hand in such simple cases.

[yavtuk]

but if you asking about JT pattern than we can call it fast pattern and golang contains the similar
out goal to add pass which will change default slow pattern (memory optimized) to fast pattern