bug: (dotnet) SO files in container are not published

[HugoVG]

Description

I'm currently using xberg and xberg-html to markdown
No problems with html to markdown. but I noticed in production xberg does not work.
I've tried debugging and fixing it on my own (without changing the package)

inside the docker container

$ ls -al /app | grep berg
-rwxr-xr-x 1 root root   976384 Jul 20 12:30 Xberg.dll
-rwxr-xr-x 1 root root 89831816 Jul 20 12:30 libxberg_ffi.so

Stack trace:

      System.DllNotFoundException: Unable to load shared library 'xberg_ffi' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable: 
      /app/xberg_ffi.so: cannot open shared object file: No such file or directory
      /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/xberg_ffi.so: cannot open shared object file: No such file or directory
      libonnxruntime.so.1: cannot open shared object file: No such file or directory
      /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/libxberg_ffi.so: cannot open shared object file: No such file or directory
      /app/xberg_ffi: cannot open shared object file: No such file or directory
      /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/xberg_ffi: cannot open shared object file: No such file or directory
      /app/libxberg_ffi: cannot open shared object file: No such file or directory
      /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/libxberg_ffi: cannot open shared object file: No such file or directory
         at Xberg.NativeMethods.ExtractInputFromBytes(IntPtr bytes, UIntPtr bytesLen, String mimeType, String filename)
         at Xberg.ExtractInput.FromBytes(Byte[] bytes, String mimeType, String filename)

Steps to reproduce

# create project to build the docker containers
aspire new aspire-starter --non-interactive --suppress-agent-init

cd aspire-starter

# add package
dotnet package add XbergIo.Xberg --project ./aspire-starter.Web/aspire-starter.Web.csproj --prerelease

# add includes
sed -i '1i using Xberg;' ./aspire-starter.Web/Program.cs

# Create the crashing code block
sed -i '/app.Run();/i var config = new ExtractionConfig\n{\n    UseCache = true,\n    EnableQualityProcessing = true,\n};\n// Will crash before it does something with the empty array\nvar result = (await XbergConverter.ExtractAsync(\n    ExtractInput.FromBytes(new byte[0], "application/pdf",\n        "empty.pdf"), config));\nvar textResult = result.Results.FirstOrDefault();' ./aspire-starter.Web/Program.cs

# Add docker
aspire add docker

sed -i '2a builder.AddDockerComposeEnvironment("env")\n    .WithDashboard(dashboard =>\n    {\n        dashboard.WithHostPort(8080)\n            .WithForwardedHeaders(enabled: true);\n    });' ./aspire-starter.Apphost/Apphost.cs

# Add docker registration
perl -i -0777 -pe 's/(\.\w+\([^)]*\));\s*(\n\s*builder\.Build\(\)\.Run\(\);)/$1.PublishAsDockerComposeService(((resource, service) =>\n    {\n        service.Name = resource.Name;\n    }));\n\n$2/gs' ./aspire-starter.Apphost/Apphost.cs

sed -i '1i using Aspire.Hosting.Docker.Resources.ServiceNodes;' ./aspire-starter.Apphost/Apphost.cs

# run aspire
aspire deploy

# docker logs now contain errors
#Unhandled exception. System.DllNotFoundException: Unable to load shared library 'xberg_ffi' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable:
#/app/xberg_ffi.so: cannot open shared object file: No such file or directory
#/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/xberg_ffi.so: cannot open shared object file: No such file or directory
#libonnxruntime.so.1: cannot open shared object file: No such file or directory
#/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/libxberg_ffi.so: cannot open shared object file: No such file or directory
#/app/xberg_ffi: cannot open shared object file: No such file or directory
#/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/xberg_ffi: cannot open shared object file: No such file or directory
#/app/libxberg_ffi: cannot open shared object file: No such file or directory
#/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.10/libxberg_ffi: cannot open shared object file: No such file or directory
#at Xberg.NativeMethods.ExtractInputFromBytes(IntPtr bytes, UIntPtr bytesLen, String mimeType, String filename)
#at Xberg.ExtractInput.FromBytes(Byte[] bytes, String mimeType, String filename)
#at Program.<Main>$(String[] args) in C:\testxberg\aspire-starter\aspire-starter.Web\Program.cs:line 51
#at Program.<Main>(String[] args)

Relevant files and configuration

[Goldziher]

Fixed in v1.0.0-rc.31.

The .so name resolution was actually fine — libxberg_ffi.so was found (as your ls -al shows). The failure was its unshipped transitive dependency libonnxruntime.so.1: a missing NEEDED library makes dlopen of libxberg_ffi.so fail, which .NET surfaces as DllNotFoundException on the top-level library.

The NuGet native packaging now bundles the full ONNX Runtime + codec closure beside libxberg_ffi.so with RUNPATH=$ORIGIN, for every Linux RID (glibc and musl) and macOS (@loader_path) — repo commit eba1ab8cf1 plus the build-csharp-natives action. So the runtime asset now carries its own ORT.

Please re-test with rc.31 or newer.

[HugoVG]

I retested with rc.31 and rc.32 and sadly it still has the issue :(
onnx runtime still didn't get along

$ ls /app/
Microsoft.Extensions.AmbientMetadata.Application.dll         Microsoft.Extensions.Telemetry.Abstractions.dll   OpenTelemetry.dll                   aspire-starter.Web
Microsoft.Extensions.Compliance.Abstractions.dll             Microsoft.Extensions.Telemetry.dll                Polly.Core.dll                      aspire-starter.Web.deps.json
Microsoft.Extensions.DependencyInjection.AutoActivation.dll  OpenTelemetry.Api.ProviderBuilderExtensions.dll   Polly.Extensions.dll                aspire-starter.Web.dll
Microsoft.Extensions.Diagnostics.ExceptionSummarization.dll  OpenTelemetry.Api.dll                             Polly.RateLimiting.dll              aspire-starter.Web.pdb
Microsoft.Extensions.Http.Diagnostics.dll                    OpenTelemetry.Exporter.OpenTelemetryProtocol.dll  Xberg.dll                           aspire-starter.Web.runtimeconfig.json
Microsoft.Extensions.Http.Resilience.dll                     OpenTelemetry.Extensions.Hosting.dll              appsettings.Development.json        aspire-starter.Web.staticwebassets.endpoints.json
Microsoft.Extensions.Resilience.dll                          OpenTelemetry.Instrumentation.AspNetCore.dll      appsettings.json                    libxberg_ffi.so
Microsoft.Extensions.ServiceDiscovery.Abstractions.dll       OpenTelemetry.Instrumentation.Http.dll            aspire-starter.ServiceDefaults.dll  wwwroot
Microsoft.Extensions.ServiceDiscovery.dll                    OpenTelemetry.Instrumentation.Runtime.dll         aspire-starter.ServiceDefaults.pdb