Skip to content

Re-use read buffer inside gRPC to reduce ingester CPU usage #3993

Open
@bboreham

Description

@bboreham

According to my profiles, half the CPU in ingesters is taken up with garbage-collection.
70% of garbage is from recvMsg in gRPC.

There is a note on the code already saying to reuse it:
https://github.com/grpc/grpc-go/blob/f320c793495fc90c222831f1708c18119793e0f8/rpc_util.go#L573-L575

I think this is hard in the general case because people can pull tricks like having the deserialised objects point into this buffer, but specifically at this point in Push() / v2Push() we know we don't retain this memory.

I suspect this will be hard to implement, but filing it anyway in case someone wants a challenge.

This will be less valuable if #1227 is implemented, because the buffers will be much smaller.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions