unable to create native thread when called piplined many times

[readyou]

Expected behavior

redis.clients.jedis.MultiNodePipelineBase#executorService should be a shared instance, do not create ExecutorService each time pipelined is called.

Actual behavior

An ExecutorService instance is created each time pipelined is called. When pipelined called many times, then exception occured:

2023-06-21 10:48:58.445 ERROR org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached] with root cause
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:798)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1343)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
at redis.clients.jedis.MultiNodePipelineBase.sync(MultiNodePipelineBase.java:132)
at com.oppo.os.ad.strategy.job.util.redis.JobRedisUtil.scanAndLoadWithPipeline(JobRedisUtil.java:287)
at com.oppo.os.ad.strategy.job.util.redis.JobRedisUtil.hScanAndLoadWithPipeline(JobRedisUtil.java:188)
at com.oppo.os.ad.strategy.job.util.redis.JobRedisUtil.lambda$scanAndLoadConcurrently$13(JobRedisUtil.java:407)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
[75.830s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.

Steps to reproduce:

@Test
void testPipeline() {
    List<String> keyList = Stream.iterate(0, i -> i + 1).map(v -> String.valueOf(v)).collect(Collectors.toList());
    for (int i = 0; i < 1000000; i++) {
        try (ClusterPipeline pipeline = jedisCluster.pipelined()) {
            for (String key : keyList) {
                pipeline.hgetAll(key);
            }
            pipeline.sync();
        }
    }
}

Redis / Jedis Configuration

Jedis version:

4.4.2

Redis version:

7.2

Java version:

11

[yangbodong22011]

Hi, @readyou The problem you mentioned has been fixed in version 4.3.3

But there are still points that can be optimized, refer to: #3333