Permalink
Cannot retrieve contributors at this time
749 lines (568 sloc)
25.4 KB
| PEP: 433 | |
| Title: Easier suppression of file descriptor inheritance | |
| Version: $Revision$ | |
| Last-Modified: $Date$ | |
| Author: Victor Stinner <vstinner@python.org> | |
| Status: Superseded | |
| Type: Standards Track | |
| Content-Type: text/x-rst | |
| Created: 10-Jan-2013 | |
| Python-Version: 3.4 | |
| Superseded-By: 446 | |
| Abstract | |
| ======== | |
| Add a new optional *cloexec* parameter on functions creating file | |
| descriptors, add different ways to change default values of this | |
| parameter, and add four new functions: | |
| * ``os.get_cloexec(fd)`` | |
| * ``os.set_cloexec(fd, cloexec=True)`` | |
| * ``sys.getdefaultcloexec()`` | |
| * ``sys.setdefaultcloexec(cloexec)`` | |
| Rationale | |
| ========= | |
| A file descriptor has a close-on-exec flag which indicates if the file | |
| descriptor will be inherited or not. | |
| On UNIX, if the close-on-exec flag is set, the file descriptor is not | |
| inherited: it will be closed at the execution of child processes; | |
| otherwise the file descriptor is inherited by child processes. | |
| On Windows, if the close-on-exec flag is set, the file descriptor is not | |
| inherited; the file descriptor is inherited by child processes if the | |
| close-on-exec flag is cleared and if ``CreateProcess()`` is called with | |
| the *bInheritHandles* parameter set to ``TRUE`` (when | |
| ``subprocess.Popen`` is created with ``close_fds=False`` for example). | |
| Windows does not have "close-on-exec" flag but an inheritance flag which | |
| is just the opposite value. For example, setting close-on-exec flag | |
| means clearing the ``HANDLE_FLAG_INHERIT`` flag of a handle. | |
| Status in Python 3.3 | |
| -------------------- | |
| On UNIX, the subprocess module closes file descriptors greater than 2 by | |
| default since Python 3.2 [#subprocess_close]_. All file descriptors | |
| created by the parent process are automatically closed in the child | |
| process. | |
| ``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of | |
| the listening socket, the parent class ``socketserver.TCPServer`` | |
| does not set this flag. | |
| There are other cases creating a subprocess or executing a new program | |
| where file descriptors are not closed: functions of the ``os.spawn*()`` | |
| and the ``os.exec*()`` families and third party modules calling | |
| ``exec()`` or ``fork()`` + ``exec()``. In this case, file descriptors | |
| are shared between the parent and the child processes which is usually | |
| unexpected and causes various issues. | |
| This PEP proposes to continue the work started with the change in the | |
| subprocess in Python 3.2, to fix the issue in any code, and not just | |
| code using subprocess. | |
| Inherited file descriptors issues | |
| --------------------------------- | |
| Closing the file descriptor in the parent process does not close the | |
| related resource (file, socket, ...) because it is still open in the | |
| child process. | |
| The listening socket of TCPServer is not closed on ``exec()``: the child | |
| process is able to get connection from new clients; if the parent closes | |
| the listening socket and create a new listening socket on the same | |
| address, it would get an "address already is used" error. | |
| Not closing file descriptors can lead to resource exhaustion: even if | |
| the parent closes all files, creating a new file descriptor may fail | |
| with "too many files" because files are still open in the child process. | |
| See also the following issues: | |
| * `Issue #2320: Race condition in subprocess using stdin | |
| <http://bugs.python.org/issue2320>`_ (2008) | |
| * `Issue #3006: subprocess.Popen causes socket to remain open after | |
| close <http://bugs.python.org/issue3006>`_ (2008) | |
| * `Issue #7213: subprocess leaks open file descriptors between Popen | |
| instances causing hangs <http://bugs.python.org/issue7213>`_ (2009) | |
| * `Issue #12786: subprocess wait() hangs when stdin is closed | |
| <http://bugs.python.org/issue12786>`_ (2011) | |
| Security | |
| -------- | |
| Leaking file descriptors is a major security vulnerability. An | |
| untrusted child process can read sensitive data like passwords and | |
| take control of the parent process though leaked file descriptors. It | |
| is for example a known vulnerability to escape from a chroot. | |
| See also the CERT recommendation: | |
| `FIO42-C. Ensure files are properly closed when they are no longer needed | |
| <https://www.securecoding.cert.org/confluence/display/seccode/FIO42-C.+Ensure+files+are+properly+closed+when+they+are+no+longer+needed>`_. | |
| Example of vulnerabilities: | |
| * `OpenSSH Security Advisory: portable-keysign-rand-helper.adv | |
| <http://www.openssh.com/txt/portable-keysign-rand-helper.adv>`_ | |
| (April 2011) | |
| * `CWE-403: Exposure of File Descriptor to Unintended Control Sphere | |
| <http://cwe.mitre.org/data/definitions/403.html>`_ (2008) | |
| * `Hijacking Apache https by mod_php | |
| <http://www.securityfocus.com/archive/1/348368>`_ (Dec 2003) | |
| * Apache: `Apr should set FD_CLOEXEC if APR_FOPEN_NOCLEANUP is not set | |
| <https://issues.apache.org/bugzilla/show_bug.cgi?id=46425>`_ | |
| (fixed in 2009) | |
| * PHP: `system() (and similar) don't cleanup opened handles of Apache | |
| <https://bugs.php.net/bug.php?id=38915>`_ (not fixed in January | |
| 2013) | |
| Atomicity | |
| --------- | |
| Using ``fcntl()`` to set the close-on-exec flag is not safe in a | |
| multithreaded application. If a thread calls ``fork()`` and ``exec()`` | |
| between the creation of the file descriptor and the call to | |
| ``fcntl(fd, F_SETFD, new_flags)``: the file descriptor will be | |
| inherited by the child process. Modern operating systems offer | |
| functions to set the flag during the creation of the file descriptor, | |
| which avoids the race condition. | |
| Portability | |
| ----------- | |
| Python 3.2 added ``socket.SOCK_CLOEXEC`` flag, Python 3.3 added | |
| ``os.O_CLOEXEC`` flag and ``os.pipe2()`` function. It is already | |
| possible to set atomically close-on-exec flag in Python 3.3 when | |
| opening a file and creating a pipe or socket. | |
| The problem is that these flags and functions are not portable: only | |
| recent versions of operating systems support them. ``O_CLOEXEC`` and | |
| ``SOCK_CLOEXEC`` flags are ignored by old Linux versions and so | |
| ``FD_CLOEXEC`` flag must be checked using ``fcntl(fd, F_GETFD)``. If | |
| the kernel ignores ``O_CLOEXEC`` or ``SOCK_CLOEXEC`` flag, a call to | |
| ``fcntl(fd, F_SETFD, flags)`` is required to set close-on-exec flag. | |
| .. note:: | |
| OpenBSD older 5.2 does not close the file descriptor with | |
| close-on-exec flag set if ``fork()`` is used before ``exec()``, but | |
| it works correctly if ``exec()`` is called without ``fork()``. Try | |
| `openbsd_bug.py <http://hg.python.org/peps/file/tip/pep-0433/openbsd_bug.py>`_. | |
| Scope | |
| ----- | |
| Applications still have to close explicitly file descriptors after a | |
| ``fork()``. The close-on-exec flag only closes file descriptors after | |
| ``exec()``, and so after ``fork()`` + ``exec()``. | |
| This PEP only change the close-on-exec flag of file descriptors | |
| created by the Python standard library, or by modules using the | |
| standard library. Third party modules not using the standard library | |
| should be modified to conform to this PEP. The new | |
| ``os.set_cloexec()`` function can be used for example. | |
| .. note:: | |
| See `Close file descriptors after fork`_ for a possible solution | |
| for ``fork()`` without ``exec()``. | |
| Proposal | |
| ======== | |
| Add a new optional *cloexec* parameter on functions creating file | |
| descriptors and different ways to change default value of this | |
| parameter. | |
| Add new functions: | |
| * ``os.get_cloexec(fd:int) -> bool``: get the | |
| close-on-exec flag of a file descriptor. Not available on all | |
| platforms. | |
| * ``os.set_cloexec(fd:int, cloexec:bool=True)``: set or clear the | |
| close-on-exec flag on a file descriptor. Not available on all | |
| platforms. | |
| * ``sys.getdefaultcloexec() -> bool``: get the current default value | |
| of the *cloexec* parameter | |
| * ``sys.setdefaultcloexec(cloexec: bool)``: set the default value | |
| of the *cloexec* parameter | |
| Add a new optional *cloexec* parameter to: | |
| * ``asyncore.dispatcher.create_socket()`` | |
| * ``io.FileIO`` | |
| * ``io.open()`` | |
| * ``open()`` | |
| * ``os.dup()`` | |
| * ``os.dup2()`` | |
| * ``os.fdopen()`` | |
| * ``os.open()`` | |
| * ``os.openpty()`` | |
| * ``os.pipe()`` | |
| * ``select.devpoll()`` | |
| * ``select.epoll()`` | |
| * ``select.kqueue()`` | |
| * ``socket.socket()`` | |
| * ``socket.socket.accept()`` | |
| * ``socket.socket.dup()`` | |
| * ``socket.socket.fromfd`` | |
| * ``socket.socketpair()`` | |
| The default value of the *cloexec* parameter is | |
| ``sys.getdefaultcloexec()``. | |
| Add a new command line option ``-e`` and an environment variable | |
| ``PYTHONCLOEXEC`` to the set close-on-exec flag by default. | |
| ``subprocess`` clears the close-on-exec flag of file descriptors of the | |
| ``pass_fds`` parameter. | |
| All functions creating file descriptors in the standard library must | |
| respect the default value of the *cloexec* parameter: | |
| ``sys.getdefaultcloexec()``. | |
| File descriptors 0 (stdin), 1 (stdout) and 2 (stderr) are expected to be | |
| inherited, but Python does not handle them differently. When | |
| ``os.dup2()`` is used to replace standard streams, ``cloexec=False`` | |
| must be specified explicitly. | |
| Drawbacks of the proposal: | |
| * It is not more possible to know if the close-on-exec flag will be | |
| set or not on a newly created file descriptor just by reading the | |
| source code. | |
| * If the inheritance of a file descriptor matters, the *cloexec* | |
| parameter must now be specified explicitly, or the library or the | |
| application will not work depending on the default value of the | |
| *cloexec* parameter. | |
| Alternatives | |
| ============ | |
| Inheritance enabled by default, default no configurable | |
| ------------------------------------------------------- | |
| Add a new optional parameter *cloexec* on functions creating file | |
| descriptors. The default value of the *cloexec* parameter is ``False``, | |
| and this default cannot be changed. File descriptor inheritance enabled by | |
| default is also the default on POSIX and on Windows. This alternative is | |
| the most conservative option. | |
| This option does not solve issues listed in the `Rationale`_ | |
| section, it only provides a helper to fix them. All functions creating | |
| file descriptors have to be modified to set *cloexec=True* in each | |
| module used by an application to fix all these issues. | |
| Inheritance enabled by default, default can only be set to True | |
| --------------------------------------------------------------- | |
| This alternative is based on the proposal: the only difference is that | |
| ``sys.setdefaultcloexec()`` does not take any argument, it can only be | |
| used to set the default value of the *cloexec* parameter to ``True``. | |
| Disable inheritance by default | |
| ------------------------------ | |
| This alternative is based on the proposal: the only difference is that | |
| the default value of the *cloexec* parameter is ``True`` (instead of | |
| ``False``). | |
| If a file must be inherited by child processes, ``cloexec=False`` | |
| parameter can be used. | |
| Advantages of setting close-on-exec flag by default: | |
| * There are far more programs that are bitten by FD inheritance upon | |
| exec (see `Inherited file descriptors issues`_ and `Security`_) | |
| than programs relying on it (see `Applications using inheritance of | |
| file descriptors`_). | |
| Drawbacks of setting close-on-exec flag by default: | |
| * It violates the principle of least surprise. Developers using the | |
| os module may expect that Python respects the POSIX standard and so | |
| that close-on-exec flag is not set by default. | |
| * The os module is written as a thin wrapper to system calls (to | |
| functions of the C standard library). If atomic flags to set | |
| close-on-exec flag are not supported (see `Appendix: Operating | |
| system support`_), a single Python function call may call 2 or 3 | |
| system calls (see `Performances`_ section). | |
| * Extra system calls, if any, may slow down Python: see | |
| `Performances`_. | |
| Backward compatibility: only a few programs rely on inheritance of file | |
| descriptors, and they only pass a few file descriptors, usually just | |
| one. These programs will fail immediately with ``EBADF`` error, and it | |
| will be simple to fix them: add ``cloexec=False`` parameter or use | |
| ``os.set_cloexec(fd, False)``. | |
| The ``subprocess`` module will be changed anyway to clear | |
| close-on-exec flag on file descriptors listed in the ``pass_fds`` | |
| parameter of Popen constructor. So it possible that these programs will | |
| not need any fix if they use the ``subprocess`` module. | |
| Close file descriptors after fork | |
| --------------------------------- | |
| This PEP does not fix issues with applications using ``fork()`` | |
| without ``exec()``. Python needs a generic process to register | |
| callbacks which would be called after a fork, see `#16500: | |
| Add an atfork module`_. Such registry could be used to close file | |
| descriptors just after a ``fork()``. | |
| Drawbacks: | |
| * It does not solve the problem on Windows: ``fork()`` does not exist | |
| on Windows | |
| * This alternative does not solve the problem for programs using | |
| ``exec()`` without ``fork()``. | |
| * A third party module may call directly the C function ``fork()`` | |
| which will not call "atfork" callbacks. | |
| * All functions creating file descriptors must be changed to register | |
| a callback and then unregister their callback when the file is | |
| closed. Or a list of *all* open file descriptors must be | |
| maintained. | |
| * The operating system is a better place than Python to close | |
| automatically file descriptors. For example, it is not easy to | |
| avoid a race condition between closing the file and unregistering | |
| the callback closing the file. | |
| open(): add "e" flag to mode | |
| ---------------------------- | |
| A new "e" mode would set close-on-exec flag (best-effort). | |
| This alternative only solves the problem for ``open()``. | |
| socket.socket() and os.pipe() do not have a ``mode`` parameter for | |
| example. | |
| Since its version 2.7, the GNU libc supports ``"e"`` flag for | |
| ``fopen()``. It uses ``O_CLOEXEC`` if available, or use ``fcntl(fd, | |
| F_SETFD, FD_CLOEXEC)``. With Visual Studio, fopen() accepts a "N" | |
| flag which uses ``O_NOINHERIT``. | |
| Bikeshedding on the name of the new parameter | |
| --------------------------------------------- | |
| * ``inherit``, ``inherited``: closer to Windows definition | |
| * ``sensitive`` | |
| * ``sterile``: "Does not produce offspring." | |
| Applications using inheritance of file descriptors | |
| ================================================== | |
| Most developers don't know that file descriptors are inherited by | |
| default. Most programs do not rely on inheritance of file descriptors. | |
| For example, ``subprocess.Popen`` was changed in Python 3.2 to close | |
| all file descriptors greater than 2 in the child process by default. | |
| No user complained about this behavior change yet. | |
| Network servers using fork may want to pass the client socket to the | |
| child process. For example, on UNIX a CGI server pass the socket | |
| client through file descriptors 0 (stdin) and 1 (stdout) using | |
| ``dup2()``. | |
| To access a restricted resource like creating a socket listening on a | |
| TCP port lower than 1024 or reading a file containing sensitive data | |
| like passwords, a common practice is: start as the root user, create a | |
| file descriptor, create a child process, drop privileges (ex: change the | |
| current user), pass the file descriptor to the child process and exit | |
| the parent process. | |
| Security is very important in such use case: leaking another file | |
| descriptor would be a critical security vulnerability (see `Security`_). | |
| The root process may not exit but monitors the child process instead, | |
| and restarts a new child process and pass the same file descriptor if | |
| the previous child process crashed. | |
| Example of programs taking file descriptors from the parent process | |
| using a command line option: | |
| * gpg: ``--status-fd <fd>``, ``--logger-fd <fd>``, etc. | |
| * openssl: ``-pass fd:<fd>`` | |
| * qemu: ``-add-fd <fd>`` | |
| * valgrind: ``--log-fd=<fd>``, ``--input-fd=<fd>``, etc. | |
| * xterm: ``-S <fd>`` | |
| On Linux, it is possible to use ``"/dev/fd/<fd>"`` filename to pass a | |
| file descriptor to a program expecting a filename. | |
| Performances | |
| ============ | |
| Setting close-on-exec flag may require additional system calls for | |
| each creation of new file descriptors. The number of additional system | |
| calls depends on the method used to set the flag: | |
| * ``O_NOINHERIT``: no additional system call | |
| * ``O_CLOEXEC``: one additional system call, but only at the creation | |
| of the first file descriptor, to check if the flag is supported. If | |
| the flag is not supported, Python has to fallback to the next method. | |
| * ``ioctl(fd, FIOCLEX)``: one additional system call per file | |
| descriptor | |
| * ``fcntl(fd, F_SETFD, flags)``: two additional system calls per file | |
| descriptor, one to get old flags and one to set new flags | |
| On Linux, setting the close-on-flag has a low overhead on performances. | |
| Results of | |
| `bench_cloexec.py <http://hg.python.org/peps/file/tip/pep-0433/bench_cloexec.py>`_ | |
| on Linux 3.6: | |
| * close-on-flag not set: 7.8 us | |
| * ``O_CLOEXEC``: 1% slower (7.9 us) | |
| * ``ioctl()``: 3% slower (8.0 us) | |
| * ``fcntl()``: 3% slower (8.0 us) | |
| Implementation | |
| ============== | |
| os.get_cloexec(fd) | |
| ------------------ | |
| Get the close-on-exec flag of a file descriptor. | |
| Pseudo-code:: | |
| if os.name == 'nt': | |
| def get_cloexec(fd): | |
| handle = _winapi._get_osfhandle(fd); | |
| flags = _winapi.GetHandleInformation(handle) | |
| return not(flags & _winapi.HANDLE_FLAG_INHERIT) | |
| else: | |
| try: | |
| import fcntl | |
| except ImportError: | |
| pass | |
| else: | |
| def get_cloexec(fd): | |
| flags = fcntl.fcntl(fd, fcntl.F_GETFD) | |
| return bool(flags & fcntl.FD_CLOEXEC) | |
| os.set_cloexec(fd, cloexec=True) | |
| -------------------------------- | |
| Set or clear the close-on-exec flag on a file descriptor. The flag | |
| is set after the creation of the file descriptor and so it is not | |
| atomic. | |
| Pseudo-code:: | |
| if os.name == 'nt': | |
| def set_cloexec(fd, cloexec=True): | |
| handle = _winapi._get_osfhandle(fd); | |
| mask = _winapi.HANDLE_FLAG_INHERIT | |
| if cloexec: | |
| flags = 0 | |
| else: | |
| flags = mask | |
| _winapi.SetHandleInformation(handle, mask, flags) | |
| else: | |
| fnctl = None | |
| ioctl = None | |
| try: | |
| import ioctl | |
| except ImportError: | |
| try: | |
| import fcntl | |
| except ImportError: | |
| pass | |
| if ioctl is not None and hasattr('FIOCLEX', ioctl): | |
| def set_cloexec(fd, cloexec=True): | |
| if cloexec: | |
| ioctl.ioctl(fd, ioctl.FIOCLEX) | |
| else: | |
| ioctl.ioctl(fd, ioctl.FIONCLEX) | |
| elif fnctl is not None: | |
| def set_cloexec(fd, cloexec=True): | |
| flags = fcntl.fcntl(fd, fcntl.F_GETFD) | |
| if cloexec: | |
| flags |= FD_CLOEXEC | |
| else: | |
| flags &= ~FD_CLOEXEC | |
| fcntl.fcntl(fd, fcntl.F_SETFD, flags) | |
| ioctl is preferred over fcntl because it requires only one syscall, | |
| instead of two syscalls for fcntl. | |
| .. note:: | |
| ``fcntl(fd, F_SETFD, flags)`` only supports one flag | |
| (``FD_CLOEXEC``), so it would be possible to avoid ``fcntl(fd, | |
| F_GETFD)``. But it may drop other flags in the future, and so it is | |
| safer to keep the two functions calls. | |
| .. note:: | |
| ``fopen()`` function of the GNU libc ignores the error if | |
| ``fcntl(fd, F_SETFD, flags)`` failed. | |
| open() | |
| ------ | |
| * Windows: ``open()`` with ``O_NOINHERIT`` flag [atomic] | |
| * ``open()`` with ``O_CLOEXEC flag`` [atomic] | |
| * ``open()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| os.dup() | |
| -------- | |
| * Windows: ``DuplicateHandle()`` [atomic] | |
| * ``fcntl(fd, F_DUPFD_CLOEXEC)`` [atomic] | |
| * ``dup()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| os.dup2() | |
| --------- | |
| * ``fcntl(fd, F_DUP2FD_CLOEXEC, fd2)`` [atomic] | |
| * ``dup3()`` with ``O_CLOEXEC`` flag [atomic] | |
| * ``dup2()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| os.pipe() | |
| --------- | |
| * Windows: ``CreatePipe()`` with | |
| ``SECURITY_ATTRIBUTES.bInheritHandle=TRUE``, or ``_pipe()`` with | |
| ``O_NOINHERIT`` flag [atomic] | |
| * ``pipe2()`` with ``O_CLOEXEC`` flag [atomic] | |
| * ``pipe()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| socket.socket() | |
| --------------- | |
| * Windows: ``WSASocket()`` with ``WSA_FLAG_NO_HANDLE_INHERIT`` flag | |
| [atomic] | |
| * ``socket()`` with ``SOCK_CLOEXEC`` flag [atomic] | |
| * ``socket()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| socket.socketpair() | |
| ------------------- | |
| * ``socketpair()`` with ``SOCK_CLOEXEC`` flag [atomic] | |
| * ``socketpair()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| socket.socket.accept() | |
| ---------------------- | |
| * ``accept4()`` with ``SOCK_CLOEXEC`` flag [atomic] | |
| * ``accept()`` + ``os.set_cloexec(fd, True)`` [best-effort] | |
| Backward compatibility | |
| ====================== | |
| There is no backward incompatible change. The default behaviour is | |
| unchanged: the close-on-exec flag is not set by default. | |
| Appendix: Operating system support | |
| ================================== | |
| Windows | |
| ------- | |
| Windows has an ``O_NOINHERIT`` flag: "Do not inherit in child | |
| processes". | |
| For example, it is supported by ``open()`` and ``_pipe()``. | |
| The flag can be cleared using | |
| ``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 0)``. | |
| ``CreateProcess()`` has an ``bInheritHandles`` parameter: if it is | |
| ``FALSE``, the handles are not inherited. If it is ``TRUE``, handles | |
| with ``HANDLE_FLAG_INHERIT`` flag set are inherited. | |
| ``subprocess.Popen`` uses ``close_fds`` option to define | |
| ``bInheritHandles``. | |
| ioctl | |
| ----- | |
| Functions: | |
| * ``ioctl(fd, FIOCLEX, 0)``: set the close-on-exec flag | |
| * ``ioctl(fd, FIONCLEX, 0)``: clear the close-on-exec flag | |
| Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD. | |
| fcntl | |
| ----- | |
| Functions: | |
| * ``flags = fcntl(fd, F_GETFD); fcntl(fd, F_SETFD, flags | FD_CLOEXEC)``: | |
| set the close-on-exec flag | |
| * ``flags = fcntl(fd, F_GETFD); fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC)``: | |
| clear the close-on-exec flag | |
| Availability: AIX, Digital UNIX, FreeBSD, HP-UX, IRIX, Linux, Mac OS | |
| X, OpenBSD, Solaris, SunOS, Unicos. | |
| Atomic flags | |
| ------------ | |
| New flags: | |
| * ``O_CLOEXEC``: available on Linux (2.6.23), FreeBSD (8.3), | |
| OpenBSD 5.0, Solaris 11, QNX, BeOS, next NetBSD release (6.1?). | |
| This flag is part of POSIX.1-2008. | |
| * ``SOCK_CLOEXEC`` flag for ``socket()`` and ``socketpair()``, | |
| available on Linux 2.6.27, OpenBSD 5.2, NetBSD 6.0. | |
| * ``WSA_FLAG_NO_HANDLE_INHERIT`` flag for ``WSASocket()``: supported | |
| on Windows 7 with SP1, Windows Server 2008 R2 with SP1, and later | |
| * ``fcntl()``: ``F_DUPFD_CLOEXEC`` flag, available on Linux 2.6.24, | |
| OpenBSD 5.0, FreeBSD 9.1, NetBSD 6.0, Solaris 11. This flag is part | |
| of POSIX.1-2008. | |
| * ``fcntl()``: ``F_DUP2FD_CLOEXEC`` flag, available on FreeBSD 9.1 | |
| and Solaris 11. | |
| * ``recvmsg()``: ``MSG_CMSG_CLOEXEC``, available on Linux 2.6.23, | |
| NetBSD 6.0. | |
| On Linux older than 2.6.23, ``O_CLOEXEC`` flag is simply ignored. So | |
| we have to check that the flag is supported by calling ``fcntl()``. If | |
| it does not work, we have to set the flag using ``ioctl()`` or | |
| ``fcntl()``. | |
| On Linux older than 2.6.27, if the ``SOCK_CLOEXEC`` flag is set in the | |
| socket type, ``socket()`` or ``socketpair()`` fail and ``errno`` is set | |
| to ``EINVAL``. | |
| On Windows XPS3, ``WSASocket()`` with ``WSAEPROTOTYPE`` when | |
| ``WSA_FLAG_NO_HANDLE_INHERIT`` flag is used. | |
| New functions: | |
| * ``dup3()``: available on Linux 2.6.27 (and glibc 2.9) | |
| * ``pipe2()``: available on Linux 2.6.27 (and glibc 2.9) | |
| * ``accept4()``: available on Linux 2.6.28 (and glibc 2.10) | |
| If ``accept4()`` is called on Linux older than 2.6.28, ``accept4()`` | |
| returns ``-1`` (fail) and ``errno`` is set to ``ENOSYS``. | |
| Links | |
| ===== | |
| Links: | |
| * `Secure File Descriptor Handling | |
| <http://udrepper.livejournal.com/20407.html>`_ (Ulrich Drepper, | |
| 2008) | |
| * `win32_support.py of the Tornado project | |
| <https://bitbucket.org/pvl/gaeseries-tornado/src/c2671cea1842/tornado/win32_support.py>`_: | |
| emulate fcntl(fd, F_SETFD, FD_CLOEXEC) using | |
| ``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)`` | |
| * `LKML: [PATCH] nextfd(2) | |
| <https://lkml.org/lkml/2012/4/1/71>`_ | |
| Python issues: | |
| * `#10115: Support accept4() for atomic setting of flags at socket | |
| creation <http://bugs.python.org/issue10115>`_ | |
| * `#12105: open() does not able to set flags, such as O_CLOEXEC | |
| <http://bugs.python.org/issue12105>`_ | |
| * `#12107: TCP listening sockets created without FD_CLOEXEC flag | |
| <http://bugs.python.org/issue12107>`_ | |
| * `#16500: Add an atfork module | |
| <http://bugs.python.org/issue16500>`_ | |
| * `#16850: Add "e" mode to open(): close-and-exec | |
| (O_CLOEXEC) / O_NOINHERIT <http://bugs.python.org/issue16850>`_ | |
| * `#16860: Use O_CLOEXEC in the tempfile module | |
| <http://bugs.python.org/issue16860>`_ | |
| * `#17036: Implementation of the PEP 433 | |
| <http://bugs.python.org/issue17036>`_ | |
| * `#16946: subprocess: _close_open_fd_range_safe() does not set | |
| close-on-exec flag on Linux < 2.6.23 if O_CLOEXEC is defined | |
| <http://bugs.python.org/issue16946>`_ | |
| * `#17070: PEP 433: Use the new cloexec to improve security and avoid | |
| bugs <http://bugs.python.org/issue17070>`_ | |
| Other languages: | |
| * Perl sets the close-on-exec flag on newly created file descriptor if | |
| their number is greater than ``$SYSTEM_FD_MAX`` (``$^F``). | |
| See `$SYSTEM_FD_MAX documentation | |
| <http://perldoc.perl.org/perlvar.html#%24SYSTEM_FD_MAX>`_. Perl does | |
| this since the creation of Perl (it was already present in Perl 1). | |
| * Ruby: `Set FD_CLOEXEC for all fds (except 0, 1, 2) | |
| <http://bugs.ruby-lang.org/issues/5041>`_ | |
| * Ruby: `O_CLOEXEC flag missing for Kernel::open | |
| <http://bugs.ruby-lang.org/issues/1291>`_: the | |
| `commit was reverted later | |
| <http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/31643>`_ | |
| * OCaml: `PR#5256: Processes opened using Unix.open_process* inherit | |
| all opened file descriptors (including sockets) | |
| <http://caml.inria.fr/mantis/view.php?id=5256>`_. OCaml has a | |
| ``Unix.set_close_on_exec`` function. | |
| Footnotes | |
| ========= | |
| .. [#subprocess_close] On UNIX since Python 3.2, subprocess.Popen() | |
| closes all file descriptors by default: ``close_fds=True``. It | |
| closes file descriptors in range 3 inclusive to ``local_max_fd`` | |
| exclusive, where ``local_max_fd`` is ``fcntl(0, F_MAXFD)`` on | |
| NetBSD, or ``sysconf(_SC_OPEN_MAX)`` otherwise. If the error pipe | |
| has a descriptor smaller than 3, ``ValueError`` is raised. | |
| Copyright | |
| ========= | |
| This document has been placed in the public domain. | |
| .. | |
| Local Variables: | |
| mode: indented-text | |
| indent-tabs-mode: nil | |
| sentence-end-double-space: t | |
| fill-column: 70 | |
| coding: utf-8 | |
| End: |