Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document if Sender is fork-safe. #60

Open
dimaqq opened this issue Sep 8, 2016 · 5 comments
Open

Document if Sender is fork-safe. #60

dimaqq opened this issue Sep 8, 2016 · 5 comments

Comments

@dimaqq
Copy link
Contributor

@dimaqq dimaqq commented Sep 8, 2016

Consider a scenario where:

  • master process attaches fluent Sender or logging Handler
  • master process forks off a bunch of worker processes
  • workers use handler

Will the socket get cloned or reopened?
Does fluent logger expect a response from fluentd?
If it does, can the response be read out by wrong sender process?

@repeatedly
Copy link
Member

@repeatedly repeatedly commented Sep 9, 2016

It depends on Python's socket implementation.
If CPython, PyPy and other implmentaions are safe, we can write "This is fork-safe."

@yangmillstheory
Copy link

@yangmillstheory yangmillstheory commented Mar 28, 2018

I'm interested in this as well, having some issues getting async handler sending to a remote fluentd host with a basic gunicorn + Django deployment

@yangmillstheory
Copy link

@yangmillstheory yangmillstheory commented Mar 31, 2018

I don't think It's definitely not fork-safe. For my example above, I needed make sure workers weren't preloaded to get the asynchandler.FluentHandler working. This is because gunicorns forks workers and the logging initialization happens pre-fork, in which a daemon is created by the async sender.

benoitc/gunicorn#1045 (comment)

@arcivanov
Copy link
Contributor

@arcivanov arcivanov commented Mar 31, 2018

Generally-speaking NO threaded application is fork-safe. Forks and threads are not compatible as POSIX mandates killing all threads upon forking. Sync sender should be safe.

@benoitc
Copy link

@benoitc benoitc commented Oct 9, 2018

If you want the fD survive you will need to make sure the worker inherit from it at the system level. This can be done similarly to what does the logging handler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.