How to concat messages handled by timeout_label

[KiranNadupuru]

Problem

I am using fluent-concat plugin with both flush_interval and timeout_label in the configuration. I am not sure how to concat the messages which are handled by timeout_label.
I cannot increase flush_interval as I am trying to figure out how to concat messages with timeout_label.

Steps to replicate

I am using below script to simulate the data generation. This will sleep 10 sec to generate the last line of the multiline message. (flush_interval is 5 sec thus getting timeout scenario)

#!/bin/sh
now=`date`
i=1
while [ $i -le $1 ]
do
now=$(date)
restart='2021-10-05 INFO-- foo00bar syslogd restart.
foo00bar syslogd shutdown.'

echo "$restart"  >> /fluentDemo/source/concatLog.log

sleep 10
echo "End of multiline message" >> /fluentDemo/source/concatLog.log
i=$(( $i + 1 ))
done

Sample data

2021-10-05 INFO-- foo00bar syslogd restart.
foo00bar syslogd shutdown.
End of multiline message
2021-10-05 INFO-- foo00bar syslogd restart.
foo00bar syslogd shutdown.
End of multiline message
2021-10-05 INFO-- foo00bar syslogd restart.
foo00bar syslogd shutdown.
End of multiline message

Below is the fluent.conf file having @concat plugin. flush_interval is 5sec, so that we can get the timeout scenario.

<source>
  @type tail
  path /fluentDemo/source/concatLog.log
  pos_file /fluentDemo/pos/concatLog.log.pos
  tag foo.concat
  read_from_head "#{ENV['READ_FROM_HEAD'] || 'false'}"
  <parse>
      @type none
    </parse>
  path_key tailed_path
</source>

<match **>
  @type relabel
  @label @NORMAL
</match>

<filter foo.concat>
  @type concat
  key message
  multiline_start_regexp /^\d{4}-\d{1,2}-\d{1,2}/
  separator ""
  flush_interval 5
  timeout_label @NORMAL
</filter>

<label @NORMAL>
<filter foo.concat>   # This second concat is to add timeout_label messages
  @type concat
  key message
  multiline_start_regexp /^\d{4}-\d{1,2}-\d{1,2}/
  separator ""
  flush_interval 5
</filter>

<match foo.**>
  @type stdout    # Expecting to print concatenated  messages
</match>
</label>

Expected Behavior

2021-12-17 07:33:38.533734513 +0000 foo.concat: {"message":"2021-10-05 INFO-- foo00bar syslogd restart.foo00bar syslogd shutdown.End of multiline message","tailed_path":"/fluentDemo/source/concatLog.log"}
2021-12-17 07:33:38.533743242 +0000 foo.concat: {"message":"2021-10-05 INFO-- foo00bar syslogd restart.foo00bar syslogd shutdown.End of multiline message","tailed_path":"/fluentDemo/source/concatLog.log"}

...

Your environment

• OS version centos-7
• fluentd --version fluentd 1.12.3
• fluent-plugin-concat (2.5.0)