How can I make an S3 bucket generate a JSON file every hour?

[susanssky]

Question

I’m not sure if I’ve done something wrong. I used the configuration below, but it ends up creating a file only every two hours, which is quite strange.
I would be really grateful if someone could help me with this. Thank you so much!

Vector Config

    s3_k8s:
      type: aws_s3
      inputs: ["k8s"]
      bucket: iam-bucket
      region: xxxxxx
      key_prefix: ""
      filename_time_format: "%Y-%m-%d-%H"
      filename_extension: json
      filename_append_uuid: false
      batch:
        timeout_secs: 3600
      auth: 
        access_key_id: XXXXXXXX
        secret_access_key: YYYYYYYY
      compression: gzip
      encoding:
        codec: json

Vector Logs

No response

[thomasqueirozb]

This seems like a bug, it doesn't look like you are doing anything incorrectly. Have you also tried using a lower timeout like 60 and verified the same behavior?

[susanssky]

I changed it to 60 as you requested, and then changed filename_time_format to "%Y-%m-%d-%H-%M".

      filename_time_format: "%Y-%m-%d-%H-%M"
      batch:
        timeout_secs: 60

The result is as follows:

To be honest, I don't quite understand why it changed from 3 PM to 9 PM...