BlockingTaskExecutor Question

[angela278]

Hi, we're currently using ServerBuilder and adding gRPC services to it like so:

builder
    .meterRegistry(meterRegistry)
    .serviceUnder("/docs", new DocService())
    .serviceUnder("/", tomcatService)
    .service(
        GrpcService.builder()
            .addServices(bindableServices)
            .useClientTimeoutHeader(false)
            .build());

We'd like to add to a blocking task executor, and were wondering what the difference between adding it to the ServerBuilder builder.blockingTaskExecutor(numThreads) versus adding it to the GrpcService builder GrpcService.builder.useBlockingTaskExecutor(true) would be. How would adding it to the ServerBuilder affect the blocking in the TomcatService?

Also where does the NUM_COMMON_BLOCKING_TASK_THREADS flag get factored in? Does the numThreads passed in builder.blockingTaskExecutor(numThreads) override it?

Finally, we noticed that the blocking task executor is by default unbounded. Is there a way for us the make it bounded, and reject requests after a certain limit?

Thank you!

[trustin]

Hi! GrpcServiceBuilder.useBlockingTaskExecutor(true) doesn't add another BlockingTaskExecutor. It merely instructs GrpcService to invoke your service implementation from the BlockingTaskExecutor you specified in ServerBuilder.blockingTaskExecutor(..). By default, unlike grpc-java, GrpcService invokes your service implementation from an event loop thread. useBlockingTaskExecutor() changes this default behavior so that it works more like grpc-java.

Also where does the NUM_COMMON_BLOCKING_TASK_THREADS flag get factored in? Does the numThreads passed in builder.blockingTaskExecutor(numThreads) override it?

There is a common BlockingTaskExecutor instance provided by CommonPools.blockingTaskExecutor(). NUM_COMMON_BLOCKING_TASK_THREADS affects CommonPools.blockingTaskExecutor()'s thread count.

If you do not call builder.blockingTaskExecutor(), your server will use CommonPools.blockingTaskExecutor(). If you do, your server will use the specified BlockingTaskExecutor or create a new one if you specified numThreads.

Finally, we noticed that the blocking task executor is by default unbounded. Is there a way for us the make it bounded, and reject requests after a certain limit?

There's a decorator called ThrottlingService, which allows you to rate-limit or reject the request based on a certain condition. You could implement a custom ThrottlingStrategy to count the number of pending requests - we don't have a ready-made implmentation at the moment unfortunately. 🥲 We have a work in progress - #4073

On the client side, there's a ConcurrencyLimitingClient, which is something we might want to port to the server side. It is not exactly what you're looking for though because it looks at the number of pending requests rather than the size of the task queue. However, the task queue size is usually proportional to the number of pending requests, right?

[angela278]

Thank you for your detailed response! Does that mean in my current code, the server is automatically using CommonPools.blockingTaskExecutor() with the default NUM_COMMON_BLOCKING_TASK_THREADS at 200? Also is there any documentation on how Armeria blocking works with Tomcat?