Permalink
Cannot retrieve contributors at this time
| PEP: 369 | |
| Title: Post import hooks | |
| Version: $Revision$ | |
| Last-Modified: $Date$ | |
| Author: Christian Heimes <christian@python.org> | |
| Status: Withdrawn | |
| Type: Standards Track | |
| Content-Type: text/x-rst | |
| Created: 02-Jan-2008 | |
| Python-Version: 2.6, 3.0 | |
| Post-History: 02-Dec-2012 | |
| Withdrawal Notice | |
| ================= | |
| This PEP has been withdrawn by its author, as much of the detailed design | |
| is no longer valid following the migration to importlib in Python 3.3. | |
| Abstract | |
| ======== | |
| This PEP proposes enhancements for the import machinery to add | |
| post import hooks. It is intended primarily to support the wider | |
| use of abstract base classes that is expected in Python 3.0. | |
| The PEP originally started as a combined PEP for lazy imports and | |
| post import hooks. After some discussion on the python-dev mailing | |
| list the PEP was parted in two separate PEPs. [1]_ | |
| Rationale | |
| ========= | |
| Python has no API to hook into the import machinery and execute code | |
| *after* a module is successfully loaded. The import hooks of PEP 302 are | |
| about finding modules and loading modules but they were not designed to | |
| as post import hooks. | |
| Use cases | |
| ========= | |
| A use case for a post import hook is mentioned in Nick Coghlan's initial | |
| posting [2]_. about callbacks on module import. It was found during the | |
| development of Python 3.0 and its ABCs. We wanted to register classes | |
| like decimal.Decimal with an ABC but the module should not be imported | |
| on every interpreter startup. Nick came up with this example:: | |
| @imp.when_imported('decimal') | |
| def register(decimal): | |
| Inexact.register(decimal.Decimal) | |
| The function ``register`` is registered as callback for the module named | |
| 'decimal'. When decimal is imported the function is called with the | |
| module object as argument. | |
| While this particular example isn't necessary in practice, (as | |
| decimal.Decimal will inherit from the appropriate abstract Number base | |
| class in 2.6 and 3.0), it still illustrates the principle. | |
| Existing implementations | |
| ======================== | |
| PJE's peak.util.imports [3]_ implements post load hooks. My | |
| implementation shares a lot with his and it's partly based on his ideas. | |
| Post import hook implementation | |
| =============================== | |
| Post import hooks are called after a module has been loaded. The hooks | |
| are callable which take one argument, the module instance. They are | |
| registered by the dotted name of the module, e.g. 'os' or 'os.path'. | |
| The callable are stored in the dict ``sys.post_import_hooks`` which | |
| is a mapping from names (as string) to a list of callables or None. | |
| States | |
| ------ | |
| No hook was registered | |
| '''''''''''''''''''''' | |
| sys.post_import_hooks contains no entry for the module | |
| A hook is registered and the module is not loaded yet | |
| ''''''''''''''''''''''''''''''''''''''''''''''''''''' | |
| The import hook registry contains an entry | |
| sys.post_import_hooks["name"] = [hook1] | |
| A module is successfully loaded | |
| ''''''''''''''''''''''''''''''' | |
| The import machinery checks if sys.post_import_hooks contains post import | |
| hooks for the newly loaded module. If hooks are found then the hooks are | |
| called in the order they were registered with the module instance as first | |
| argument. The processing of the hooks is stopped when a method raises an | |
| exception. At the end the entry for the module name set to None, even | |
| when an error has occurred. | |
| Additionally the new ``__notified__`` slot of the module object is set | |
| to ``True`` in order to prevent infinity recursions when the notification | |
| method is called inside a hook. For object which don't subclass from | |
| ``PyModule`` a new attribute is added instead. | |
| A module can't be loaded | |
| '''''''''''''''''''''''' | |
| The import hooks are neither called nor removed from the registry. It | |
| may be possible to load the module later. | |
| A hook is registered but the module is already loaded | |
| ''''''''''''''''''''''''''''''''''''''''''''''''''''' | |
| The hook is fired immediately. | |
| Invariants | |
| ---------- | |
| The import hook system guarantees certain invariants. XXX | |
| Sample Python implementation | |
| ---------------------------- | |
| A Python implementation may look like:: | |
| def notify(name): | |
| try: | |
| module = sys.modules[name] | |
| except KeyError: | |
| raise ImportError("Module %s has not been imported" % (name,)) | |
| if module.__notified__: | |
| return | |
| try: | |
| module.__notified__ = True | |
| if '.' in name: | |
| notify(name[:name.rfind('.')]) | |
| for callback in post_import_hooks[name]: | |
| callback(module) | |
| finally: | |
| post_import_hooks[name] = None | |
| XXX | |
| C API | |
| ----- | |
| New C API functions | |
| ''''''''''''''''''' | |
| ``PyObject* PyImport_GetPostImportHooks(void)`` | |
| Returns the dict sys.post_import_hooks or NULL | |
| ``PyObject* PyImport_NotifyLoadedByModule(PyObject *module)`` | |
| Notify the post import system that a module was requested. Returns the | |
| a borrowed reference to the same module object or NULL if an error has | |
| occurred. The function calls only the hooks for the module itself and not | |
| its parents. The function must be called with the import lock acquired. | |
| ``PyObject* PyImport_NotifyLoadedByName(const char *name)`` | |
| ``PyImport_NotifyLoadedByName("a.b.c")`` calls | |
| ``PyImport_NotifyLoadedByModule()`` for ``a``, ``a.b`` and ``a.b.c`` | |
| in that particular order. The modules are retrieved from | |
| ``sys.modules``. If a module can't be retrieved, an exception is raised | |
| otherwise the a borrowed reference to ``modname`` is returned. | |
| The hook calls always start with the prime parent module. | |
| The caller of PyImport_NotifyLoadedByName() must hold the import lock! | |
| ``PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name)`` | |
| Register a new hook ``callable`` for the module ``mod_name`` | |
| ``int PyModule_GetNotified(PyObject *module)`` | |
| Returns the status of the ``__notified__`` slot / attribute. | |
| ``int PyModule_SetNotified(PyObject *module, int status)`` | |
| Set the status of the ``__notified__`` slot / attribute. | |
| The ``PyImport_NotifyLoadedByModule()`` method is called inside | |
| ``import_submodule()``. The import system makes sure that the import lock | |
| is acquired and the hooks for the parent modules are already called. | |
| Python API | |
| ---------- | |
| The import hook registry and two new API methods are exposed through the | |
| ``sys`` and ``imp`` module. | |
| ``sys.post_import_hooks`` | |
| The dict contains the post import hooks:: | |
| {"name" : [hook1, hook2], ...} | |
| ``imp.register_post_import_hook(hook: "callable", name: str)`` | |
| Register a new hook *hook* for the module *name* | |
| ``imp.notify_module_loaded(module: "module instance") -> module`` | |
| Notify the system that a module has been loaded. The method is provided | |
| for compatibility with existing lazy / deferred import extensions. | |
| ``module.__notified__`` | |
| A slot of a module instance. XXX | |
| The when_imported function decorator is also in the imp module, | |
| which is equivalent to:: | |
| def when_imported(name): | |
| def register(hook): | |
| register_post_import_hook(hook, name) | |
| return register | |
| imp.when_imported(name) -> decorator function | |
| for @when_imported(name) def hook(module): pass | |
| Open issues | |
| =========== | |
| The when_imported decorator hasn't been written. | |
| The code contains several XXX comments. They are mostly about error | |
| handling in edge cases. | |
| Backwards Compatibility | |
| ======================= | |
| The new features and API don't conflict with old import system of Python | |
| and don't cause any backward compatibility issues for most software. | |
| However systems like PEAK and Zope which implement their own lazy import | |
| magic need to follow some rules. | |
| The post import hooks carefully designed to cooperate with existing | |
| deferred and lazy import systems. It's the suggestion of the PEP author | |
| to replace own on-load-hooks with the new hook API. The alternative | |
| lazy or deferred imports will still work but the implementations must | |
| call the ``imp.notify_module_loaded`` function. | |
| Reference Implementation | |
| ======================== | |
| A reference implementation is already written and is available in the | |
| *py3k-importhook* branch. [4]_ It still requires some cleanups, | |
| documentation updates and additional unit tests. | |
| Acknowledgments | |
| =============== | |
| Nick Coghlan, for proof reading and the initial discussion | |
| Phillip J. Eby, for his implementation in PEAK and help with my own implementation | |
| Copyright | |
| ========= | |
| This document has been placed in the public domain. | |
| References | |
| ========== | |
| .. [1] PEP: Lazy module imports and post import hook | |
| http://permalink.gmane.org/gmane.comp.python.devel/90949 | |
| .. [2] Interest in PEP for callbacks on module import | |
| http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126 | |
| .. [3] peak.utils.imports | |
| http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup | |
| .. [4] py3k-importhook branch | |
| http://svn.python.org/view/python/branches/py3k-importhook/ | |
| .. | |
| Local Variables: | |
| mode: indented-text | |
| indent-tabs-mode: nil | |
| sentence-end-double-space: t | |
| fill-column: 70 | |
| coding: utf-8 | |
| End: |