New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bpo-21736: Set __file__ on frozen stdlib modules. #28656
bpo-21736: Set __file__ on frozen stdlib modules. #28656
Conversation
a1cc171
to
36112c1
Compare
|
If you want to schedule another build, you need to add the " |
7934888
to
1d6ff36
Compare
|
If you want to schedule another build, you need to add the " |
1d6ff36
to
d8e19e0
Compare
|
If you want to schedule another build, you need to add the " |
c9a77db
to
9687202
Compare
|
This appears to have caused https://bugs.python.org/issue45506 - out of tree builds can't pass several tests. |
Currently frozen modules do not have
__file__set. In their spec,originis set to "frozen" and they are marked as not having a location. (Similarly, for frozen packages__path__is set to an empty list.) However, for frozen stdlib modules we are able to extrapolate__file__as long as we can determine the stdlib directory at runtime. (We now do so since gh-28586.) Having__file__set is helpful for a number of reasons. Likewise, having a non-empty__path__means we can import submodules of a frozen package from the filesystem (e.g. we could partially freeze theencodingsmodule).This change sets
__file__(and adds to__path__) for frozen stdlib modules. It usessys._stdlibdir(from gh-28586) and the frozen module alias information (from gh-28655). All that work is done inFrozenImporter(inLib/importlib/_bootstrap.py). Also, if a frozen module is imported before importlib is bootstrapped (during interpreter initialization) then we fix up that module and its spec during the importlib bootstrapping step (i.e.imporlib._bootstrap._setup()) to match what gets set byFrozenImporter, including setting the file info (if the stdlib dir is known). To facilitate this, modules imported usingPyImport_ImportFrozenModule()have__origname__set using the frozen module alias info.__origname__is popped off during importlib bootstrap.(To be clear, even with this PR the new code to set
__file__during fixups inimporlib._bootstrap._setup()doesn't actually get triggered yet. This is becausesys._stdlibdirhasn't been set yet in interpreter initialization at the point importlib is bootstrapped. However, we do fix up such modules at that point to otherwise match the result of importing throughFrozenImporter, just not the__file__and__path__parts. Doing so will require changes in the order in which things happen during interpreter initialization. That can be addressed separately. Once it is, the file-related fixup code from this PR will kick in.)Here are things this PR does not do:
__file__for non-stdlib modules (no way of knowing the parent dir)__file__if the stdlib dir is not known (nor assume the expense of finding it)__file__if the stdlib is in a zip file__file__actually exists (too expensive)__path__for frozen packages that alias a non-package (since there is no package dir)Other things this PR skips, but we may do later:
__file__on modules imported usingPyImport_ImportFrozenModule()co_filenamewhen we unmarshal the frozen code object while importing the module (e.g. inFrozenImporter.exec_module()) -- this would allow tracebacks to show source linesFrozenImporter.get_filename()andFrozenImporter.get_source()https://bugs.python.org/issue21736