add support to manage logs using lumber jack rolling logger

[sushanth0910]

What type of PR is this?

Adding support to go runner to use lumber jack logger for log rotation.

What this PR does / why we need it:

As per the linux log rotation we have copy truncate and create mechanisms to handle log rotation.

Copy truncate mechanism first copies logs to a backup file then start truncating the original file. During the truncate process our components/application still continues to wright to old file which will be truncated but never copied to the backup file.

Amount of logs lost is depends on file size during log rotation. If log rotation takes 10 seconds to truncate the log file component/application logs are lost for that 10 seconds. Using Copy truncate mechanism we might loose data as part of log rotation process. It was also mentioned in https://linux.die.net/man/8/logrotate.

Create mechanism renames old log file and creates new log file as part of log rotation process. For the application/component to start writing to the new log file we have to either

• restart the application/component (we dont want to restart components too many time)
• or send some kind of signal (like HUP) to the application/component to reopen the log file(which requires code change to handle log file reopen on all kubernetes components running on top of go runner).

Failing to do so the application continues to write logs to the old log file(renamed log file). Both of these mechanisms has some kind of issue to be taken care of.

This PR introduces using lumber jack logger as a option to handle log rotation with go runner.
k8s upstream also use lumber jack to handle audlit logs
using lumberjack once current log file reaches ts max-size limit specified by user Lumberjack renames log file and starts writing to new log file without dropping any logs. This way we will not loose logs as part of log rotation.

Which issue(s) this PR fixes:

None

Special notes for your reviewer:

Does this PR introduce a user-facing change?

This PR doesn't directly modify current behavior. New change is added as an option. Users can enable it using below flags.

New flag added. User can set below flags if they want go-runner to handle log rotation using lumberjack logger
    - --log-rotate=true
    - --log-maxsize=100
    - --log-maxbackup=5
    - --log-compress=true

[k8s-ci-robot]

Welcome @sushanth0910!

It looks like this is your first PR to kubernetes/release 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/release has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @sushanth0910. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sushanth0910
Once this PR has been reviewed and has the lgtm label, please assign puerco for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• images/build/go-runner/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[saschagrunert]

We need to reach consensus if we want to introduce a dependency to go-runner. I'm not in favor of that.

/hold

[saschagrunert]

cc @kubernetes/release-managers @kubernetes/sig-architecture

[xmudrii]

Since go-runner is used for various core Kubernetes components, maybe we should consider requiring a KEP for such a change.

[liggitt]

I really do not want to see us growing dependencies outside the stdlib for this

[jeremyrickard]

@xmudrii is correct this is used for basically all the components so at a minimum we'd need to think about this beyond the scope of just this PR.

Overall -1, in agreement with @liggitt and @saschagrunert

[cpanato]

+1 with sascha and liggitt comments

this might be achieve using a sidecar if need to do that

[dims]

thanks for all the feedback folks!

[prateekgogia]

I really do not want to see us growing dependencies outside the stdlib for this

Looking at the Lumberjack go.mod file there are no third party dependencies. At this point its only using all stdlib code and has been pretty stable for last few years (used in APIserver audit logs as well).

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle rotten
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.