After updating our project to use snappy-java v1.1.10.8, some of our users reported an issue when starting the process:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
	at tech.pegasys.teku.networking.eth2.gossip.encoding.SnappyBlockCompressor.compress(SnappyBlockCompressor.java:54) ~[teku-networking-eth2-25.9.0.jar:25.9.0]
	at tech.pegasys.teku.networking.eth2.gossip.encoding.SszSnappyEncoding.encode(SszSnappyEncoding.java:38) ~[teku-networking-eth2-25.9.0.jar:25.9.0]
	at tech.pegasys.teku.networking.eth2.gossip.subnets.AttestationSubnetSubscriptions.lambda$gossip$0(AttestationSubnetSubscriptions.java:90) ~[teku-networking-eth2-25.9.0.jar:25.9.0]
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[?:?]
	... 46 more
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.10-5a7a1634-b5ef-475e-89cd-db2863cbec50-libsnappyjava.so: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory

After looking through v1.1.10.8 changelog, I noticed the following change: #632

This change introduced a new code path that checks for the existence of /lib/ld-musl-x86_64.so.1 (https://github.com/xerial/snappy-java/blob/main/src/main/java/org/xerial/snappy/OSInfo.java#L238-L244). However, it looks like during runtime, libsnappyjava.so is trying to load libc.musl-x86_64.so.1, and it fails.

I am not sure of the reason for this mismatch. It feels unintentional but maybe I am just misinformed. Either way, we have found a few workarounds.

1. We can create a symlink libc.musl-x86_64.so.1 -> /lib/ld-musl-x86_64.so.1; or
2. We can just completely uninstall musl.

Workaround 1 seems a bit hacky so we won't be asking our users to do so. Workaround 2 isn't great either, because musl is a dependency of grafana, and most of our users run a grafana instance on their nodes as well to monitor our process.

We are going to downgrade snappy-java to v1.1.10.7 in the mean time, until we have more information on this issue.