[SYCL][A64FX] Building SYCL on A64FX

[HariAbram]

I am trying to build SYCL on the A64FX machine, I could compile and link POCL and SYCL on A64FX. SYCL code compiles without a problem, but I am encountering runtime issues. Below I am attaching both SYCL trace and POCL debug

POCL debug trace:

`$ POCL_DEBUG=all ./a.out 
** Final POCL_DEBUG flags: FFFFFFFFFFFFFFFF 
[2024-04-24 08:19:06.950667700]POCL: in fn pocl_read_file at line 173:
  |     ERROR |  fopen( /sys/devices/system/cpu/cpu0/cache/index3/size ) failed
[2024-04-24 08:19:06.950818591]POCL: in fn pocl_read_file at line 173:
  |     ERROR |  fopen( /sys/devices/system/cpu/cpu0/cache/index2/size ) failed
[2024-04-24 08:19:06.281471632590422]POCL: in fn pocl_topology_detect_device_info at line 244:
  |   WARNING |  Could not figure out CPU cache size, using bogus value
[2024-04-24 08:19:06.281471640175367]POCL: in fn POclCreateCommandQueue at line 104:
  |   GENERAL |  Created Command Queue 3 (0x24c90f30) on device 0
[2024-04-24 08:19:06.473404917498]POCL: in fn void pocl_llvm_create_context(cl_context) at line 435:
  |      LLVM |  Created context 2 (0x24c902e0)
[2024-04-24 08:19:06.4616194021354778878]POCL: in fn POclCreateContext at line 233:
  |   GENERAL |  Created Context 2 (0x24c902e0)
[2024-04-24 08:19:06.281471640334249]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0x24c902e0), Refcount: 2
[2024-04-24 08:19:06.281471640362739]POCL: in fn POclCreateCommandQueue at line 104:
  |   GENERAL |  Created Command Queue 4 (0x24c933c0) on device 0
[2024-04-24 08:19:06.958725800]POCL: in fn pocl_basic_usm_alloc at line 946:
  |   WARNING |  ALLOCATED: ALL_PTR: 0x24c93780 ALL_SIZE: 40
[2024-04-24 08:19:06.958772441]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0x24c902e0), Refcount: 3
[2024-04-24 08:19:06.4503600586168197]POCL: in fn pocl_usm_alloc at line 134:
  |    MEMORY |  Allocated USM: PTR 0x24c93780, SIZE 40, FLAGS 0 
[2024-04-24 08:19:06.958838471]POCL: in fn pocl_basic_usm_alloc at line 946:
  |   WARNING |  ALLOCATED: ALL_PTR: 0x24c93980 ALL_SIZE: 40
[2024-04-24 08:19:06.958864662]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0x24c902e0), Refcount: 4
[2024-04-24 08:19:06.4503600586258228]POCL: in fn pocl_usm_alloc at line 134:
  |    MEMORY |  Allocated USM: PTR 0x24c93980, SIZE 40, FLAGS 0 
[2024-04-24 08:19:06.281471640683492]POCL: in fn POclRetainCommandQueue at line 35:
  | REFCOUNTS |  Retain Command Queue 4 (0x24c933c0), Refcount: 2
[2024-04-24 08:19:06.4503600586339989]POCL: in fn pocl_create_event at line 528:
  |    EVENTS |  Created event 1 (0x24c93d70) Command unknown
[2024-04-24 08:19:06.281471640736613]POCL: in fn pocl_create_command_struct at line 647:
  |    EVENTS |  event pointer provided
[2024-04-24 08:19:06.281471640757863]POCL: in fn pocl_create_command_struct at line 670:
  |    EVENTS |  Created immediate command struct: CMD 0x24c93c60 (event 1 / 0x24c93d70, type: unknown)
[2024-04-24 08:19:06.959038363]POCL: in fn pocl_command_enqueue at line 1235:
  |    EVENTS |  Pushed Event 1 to CQ 4.
[2024-04-24 08:19:06.2676586395344707082]POCL: in fn pocl_update_event_queued at line 2086:
  |    EVENTS |  Event queued: 1
[2024-04-24 08:19:06.281471640838064]POCL: in fn pocl_update_event_submitted at line 2106:
  |    EVENTS |  Event submitted: 1
[2024-04-24 08:19:06.959303746]POCL: in fn pocl_update_event_running_unlocked at line 2125:
  |    EVENTS |  Event running: 1
[2024-04-24 08:19:06.959355616]POCL: in fn pocl_update_event_finished at line 2219:
  |    EVENTS |  cpu: Command complete, event 1
[2024-04-24 08:19:06.959384527]POCL: in fn pocl_exec_command at line 694:
  |    TIMING |       >>>        52.040  us    Event SVM Memcpy            
[2024-04-24 08:19:06.281471641160107]POCL: in fn POclReleaseEvent at line 40:
  | REFCOUNTS |  Release Event 1 (0x24c93d70), Refcount: 1
[2024-04-24 08:19:06.281471641202527]POCL: in fn POclReleaseEvent at line 40:
  | REFCOUNTS |  Release Event 1 (0x24c93d70), Refcount: 0
[2024-04-24 08:19:06.959499648]POCL: in fn POclReleaseEvent at line 65:
  | REFCOUNTS |  Free Event 1 (0x24c93d70)
[2024-04-24 08:19:06.959532958]POCL: in fn POclReleaseCommandQueue at line 43:
  | REFCOUNTS |  Release Command Queue 4 (0x24c933c0), Refcount: 1
[2024-04-24 08:19:06.959745450]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0x24c902e0), Refcount: 5
[2024-04-24 08:19:06.5256122140]POCL: in fn pocl_run_command_capture_output at line 2007:
  |   GENERAL |  Launching: /home/hari.abram/sycl_workspace/llvm/build/install/bin/llvm-spirv
[2024-04-24 08:19:07.014287944]POCL: in fn compile_and_link_program at line 708:
  |      LLVM |  building program with options 
[2024-04-24 08:19:07.014334324]POCL: in fn compile_and_link_program at line 745:
  |      LLVM |  building program for 1 devs with options 
[2024-04-24 08:19:07.014360815]POCL: in fn compile_and_link_program at line 749:
  |      LLVM |     BUILDING for device: cpu
[2024-04-24 08:19:07.015574367]POCL: in fn pocl_llvm_convert_and_link_ir at line 635:
  |      LLVM |  Cached compiled SPIRV binary not found, generating SPIR IR to /home/hari.abram/.cache/pocl/kcache/EB/MLKFABANFNAFPACPNFILLLKOOGAGHBCLLMBNP/program.bc
[2024-04-24 08:19:07.281470698552959]POCL: in fn pocl_run_command at line 1956:
  |   GENERAL |  Launching: /home/hari.abram/sycl_workspace/llvm/build/install/bin/llvm-spirv
[2024-04-24 08:19:07.281470758908421]POCL: in fn llvm::Module *getKernelLibrary(cl_device_id, PoclLLVMContextData *) at line 1067:
  |      LLVM |  Using /home/hari.abram/POCL/install/pocl/lib64/../share/pocl/kernel-aarch64-linux-gnu-a64fx.bc as the built-in lib.
[2024-04-24 08:19:08.780434948]POCL: in fn int pocl_llvm_link_program(cl_program, unsigned int, cl_uint, unsigned char **, size_t *, void **, int, int) at line 858:
  |     ERROR | CL_LINK_PROGRAM_FAILURE could not parse module
[2024-04-24 08:19:08.281471462239819]POCL: in fn pocl_llvm_convert_and_link_ir at line 693:
  |     ERROR | CL_LINK_PROGRAM_FAILURE Failed to link program.bc
[2024-04-24 08:19:08.780529579]POCL: in fn compile_and_link_program at line 874:
  |     ERROR | CL_BUILD_PROGRAM_FAILURE Device cpu-0x001 failed to build the program
[2024-04-24 08:19:08.281471462490641]POCL: in fn POclReleaseProgram at line 52:
  | REFCOUNTS |  Release Program 6 (0x24c958a0), Refcount: 0, Kernel #: 0 
[2024-04-24 08:19:08.780780192]POCL: in fn POclReleaseProgram at line 62:
  | REFCOUNTS |  Free Program 6 (0x24c958a0)
[2024-04-24 08:19:08.780891123]POCL: in fn POclReleaseContext at line 49:
  | REFCOUNTS |  Release Context 2 (0x24c902e0), Refcount: 4
terminate called after throwing an instance of 'sycl::_V1::compile_program_error'
  what():  The program was built for 1 devices
Build program log for 'cpu-0x001':
Device cpu-0x001 failed to build the program
 -11 (PI_ERROR_BUILD_PROGRAM_FAILURE)
Aborted (core dumped)`

SYCL trace with SYCL_PI_TRACE=-1

`SYCL_PI_TRACE[-1]: dlopen(/home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_cuda.so) failed with </home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_cuda.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[-1]: dlopen(/home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_hip.so) failed with </home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_hip.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[-1]: dlopen(/home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_native_cpu.so) failed with </home/hari.abram/sycl_workspace/llvm/build/install/lib/libpi_native_cpu.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 15.48.1 ]
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 15.48.1 ]
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_cuda.so
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_unified_runtime.so [ PluginVersion: 15.48.1 ]
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_native_cpu.so
terminate called after throwing an instance of 'sycl::_V1::compile_program_error'
  what():  The program was built for 1 devices
Build program log for 'cpu-0x001':
Device cpu-0x001 failed to build the program
 -11 (PI_ERROR_BUILD_PROGRAM_FAILURE)
Aborted (core dumped)`

Did anyone had this problem? How do I solve it?

[steffenlarsen]

Hi @HariAbram! What CPU and which version of POCL are you using?

[HariAbram]

I am using Fujitsu A64FX (ARM AArch64 architecture), with POCL v4. I tested POCL separately, with standard examples, and it is working fine.