Open
Description
According to my profiles, half the CPU in ingesters is taken up with garbage-collection.
70% of garbage is from recvMsg
in gRPC.
There is a note on the code already saying to reuse it:
https://github.com/grpc/grpc-go/blob/f320c793495fc90c222831f1708c18119793e0f8/rpc_util.go#L573-L575
I think this is hard in the general case because people can pull tricks like having the deserialised objects point into this buffer, but specifically at this point in Push()
/ v2Push()
we know we don't retain this memory.
I suspect this will be hard to implement, but filing it anyway in case someone wants a challenge.
This will be less valuable if #1227 is implemented, because the buffers will be much smaller.