Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

structured logging: replacing Fatal/Exit/etc. without loss of flushing #303

Merged
merged 1 commit into from Mar 14, 2022

Conversation

pohly
Copy link

@pohly pohly commented Mar 11, 2022

What this PR does / why we need it:

When structured logging called for functions like Fatal and Exit to be avoided
in favor of klog.ErrorS + os.Exit, it was overlooked that this replacement
skips flushing.

We could ask developers to use klog.ErrorS + klog.Flush + os.Exit, but that is
still not quite the same because klog.Flush could get stuck or take too long.

Therefore klog now supports flushing + exiting with a dedicated function that
can be combined with some other, arbitrary logging function.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #301

Special notes for your reviewer:

See also https://kubernetes.slack.com/archives/C020CCMUEAX/p1644341190882399

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

When replacing calls like Fatal or Exit, use FlushAndExit instead of os.Exit to ensure that log data gets flushed.

@dims
Copy link
Member

@dims dims commented Mar 11, 2022

exit.go Show resolved Hide resolved
"k8s.io/klog/v2"
)

func ExampleFlushAndExit() {
Copy link

@pchan pchan Mar 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why we can't have a TestFlushAndExit (Similar to functions in TestErrorS in klog_test.go) ?

We can check against the following format.

format: "\"exiting...\"\nos.Exit(10)"

Copy link
Author

@pohly pohly Mar 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have such a test function, but an example is more useful:

  • it gets tested by go test
  • it gets included in the documentation

@pohly
Copy link
Author

@pohly pohly commented Mar 12, 2022

@pchan I've pushed an update, please take another look.

@pchan
Copy link

@pchan pchan commented Mar 13, 2022

@pchan I've pushed an update, please take another look.

LGTM.

@pohly
Copy link
Author

@pohly pohly commented Mar 13, 2022

/assign @dims

For approval + final LGTM.

// Applications that replace those calls and do not have some specific
// requirements like "exit immediately" can use this value as parameter
// for FlushAndExit.
ExitFlushTimeout = 10 * time.Second
Copy link
Member

@dims dims Mar 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are end users able to modify these directly? they shouldn't right?

Copy link
Author

@pohly pohly Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a bit need for modifying it, but also no good reason why we shouldn't allow it. Being able to modify them is mostly (just?) useful in tests.

This timeout can become a constant.

The OsExit function has to be a variable, but we could unexport it.

Copy link
Author

@pohly pohly Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking of OsExit, if we allow modifying it, should Fatal and friends also call it?

There is a feature request where someone is asking for Fatal to not call os.Exit. We cannot change the default behavior and not call os.Exit (that behavior is part of the stable API), but if we export OsExit and use it in Fatal, then someone who knows his application can perhaps change it into a function which panics.

Copy link
Author

@pohly pohly Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed a version that uses OsExit everywhere instead of calling os.Exit directly and also updated the comments: https://github.com/kubernetes/klog/compare/dbc5aa051e765a4216e0fb89e0b54fae92ad8ab0..56b8a4ea35ba0fc3be9fc82182e71c2f0116adc1

When structured logging called for functions like Fatal and Exit to be avoided
in favor of klog.ErrorS + os.Exit, it was overlooked that this replacement
skips flushing.

We could ask developers to use klog.ErrorS + klog.Flush + os.Exit, but that is
still not quite the same because klog.Flush could get stuck or take too long.
Therefore klog now supports flushing + exiting with a dedicated function that
can be combined with some other, arbitrary logging function.

Allowing applications to change the default timeout and how specifically the
application terminates may be useful, so both values are variables that are
exported and users of klog are allowed to change.
@serathius
Copy link

@serathius serathius commented Mar 14, 2022

/lgtm

/hold
for @dims review

@dims
Copy link
Member

@dims dims commented Mar 14, 2022

/hold cancel

/approve
/lgtm

@k8s-ci-robot
Copy link

@k8s-ci-robot k8s-ci-robot commented Mar 14, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, pohly, serathius

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f676d4a into kubernetes:main Mar 14, 2022
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

6 participants