You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As #12572 has discussed, ClassCircularityError caused by ThreadLocalRandom has been avoided by class preloading. This prevents 99% of our ClassCircularityError that happens with the ThreadLocalRandom class (For better understanding, you may refer to this post in the ByteBuddy community) However, I've got a new ClassCircularityError to our attention. Per stack trace in the skywalking agent log (not in the application's log output this time) and heap analysis, this ClassCircularityError happened with RetriableStream$1CommitTask, at its instantiation. StackTrace below
java.lang.ClassCircularityError: org/apache/skywalking/apm/dependencies/io/grpc/internal/RetriableStream$1CommitTask
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStrean.commit (RetriableStream.java: 206)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream.commit.AndRun (RetriableStream.java: 216)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream.access$1400 (RetriableStream.java: 55)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream$Sublistener.closed (RetriableStream.java: 979)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.FailingClientStream.start (FailingClientStream.java: 61)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream$StartEntry.runWith(RetriableStream.java: 349)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream.drain (RetriableStream.java: 313)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.RetriableStream.start (RetriableStream.java: 390)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.ClientCallImpl.startInternal (ClientCallImp1.java: 289)
at org.apache.skywalking.apm.dependencies.io.grpc.internal.ClientCallImpl.start (ClientCallImpl.java: 191)
at org.apache.skywalking.apm.dependencies.io.grpc.ForwardingClientCall.start (ForwardingClientCall.java: 32)
at org.apache.skywalking.apm.agent. core.remote.AgentIDDecorator$1$1.start (AgentIDDecorator.java: 48)
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.startCall (ClientCalls.java: 341)
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.asyncUnaryRequestCall (ClientCalls.java: 315)
at org.apache.skywalking.apm.dependenciez.io.grpc.stub.ClientCalls.futureUnaryCall (ClientCalls.java: 227)
at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.blockingUnaryCall (ClientCalls.java: 154)
at org.apache.skywalking.apm.network.language.profile.v3.ProfileTaskGrpc$ProfileTaskBlockingStub.getProfileTaskCommands(ProfileTaskGrpc.java: 284)
at org.apache.skywalking.apm.agent.core.profile.ProfileTaskChannelService.run(ProfileTaskChannelService.java: 90)
at org.apache.skywalking.apm.util.FurnableWithExceptionProtection.run(RunnableWithExceptionProtection.java: 33)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java: 611)
......
We checked io.grpc:grpc-core:1.50.0 RetriableStream.java: 206
This exception caused CommitTask not to be executed and further corresponding RetryStream not to be removed from the UncommitedRetriableStream map, resulting in a map with a giant number of RetryStream objects, causing a constant increase of GC cost and thus RT.
Considering this happens during the execution of SkyWalking agent core function, I want to bring this out here and ask for any opinion or advice to handle this.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
As #12572 has discussed, ClassCircularityError caused by
ThreadLocalRandom
has been avoided by class preloading. This prevents 99% of our ClassCircularityError that happens with theThreadLocalRandom
class (For better understanding, you may refer to this post in the ByteBuddy community) However, I've got a new ClassCircularityError to our attention. Per stack trace in the skywalking agent log (not in the application's log output this time) and heap analysis, this ClassCircularityError happened withRetriableStream$1CommitTask
, at its instantiation. StackTrace belowWe checked io.grpc:grpc-core:1.50.0 RetriableStream.java: 206
This exception caused
CommitTask
not to be executed and further corresponding RetryStream not to be removed from theUncommitedRetriableStream
map, resulting in a map with a giant number ofRetryStream
objects, causing a constant increase of GC cost and thus RT.Considering this happens during the execution of SkyWalking agent core function, I want to bring this out here and ask for any opinion or advice to handle this.
Beta Was this translation helpful? Give feedback.
All reactions