Swift: generalize open redirection on both platforms and rework output rewriting#11571
Conversation
41b6d2a to
1b48dbf
Compare
The hash map mechanism that was already in use for reading swiftmodule files on macOS is now in use also on Linux. The output replacing mechanism has been also reworked so that: * frontend module emission modes have the remapping done directly in the internal frontend options instead of painstakingly modifying input flags (this requires a patch on the swift headers though) * object emission mode is silenced to be just a type checking pass, thus producing no output files * all other passes but some debugging and version related ones become noops The open file read redirection uses a global weak pointer instance to maximize robustness in the face of possibly multi-threaded calls to open happening while `main` is exiting. Possibly overkill, but better safe than sorry.
be5e084 to
fd2d16c
Compare
Downgrading the emit object action to a type check one has some unexpected side effects, that seem however acceptable: * experimental false static assertions do not make compilation fail in type check mode * the implicit module loading of `SwiftOnoneSupport` is not happening. That module contains some "pre-specializations", it does not seem really relevant for analysis
fd2d16c to
26ae8f1
Compare
|
Tested it locally against two big projects - works great! Let's wait for DCA and merge? |
Thanks for that! I aborted the latest expriment as I wanted to add 7162692 in before that. I'll be restarting the experiment right away. I wonder whether this has a good impact on performance, as we probably avoid a significant amount of compiler work duplication with this. |
This reverts commit 26ae8f1.
Also, handle the corner case where loading libc fails.
342c87b to
208388e
Compare
andersfugmann
left a comment
There was a problem hiding this comment.
Great work on unifying artifact handling across platforms!
Just a drive by commenting - I did not do a full review.
andersfugmann
left a comment
There was a problem hiding this comment.
Great work on unifying artifact handling across platforms!
Just a drive by commenting - I did not do a full review.
File hashing is now done internally in `SwiftFileInterception` (and exported as a `getHashOfRealFile` function for future use in linkage awareness), and using a per-process in-memory cache. The persistent caching of paths is removed, so the solution is now robust against input file changes during the build. For the same reason, the hash to artifact mapping have the symlinks reversed now. The artifacts themselves are stored using the hash as filenames, and the original paths of the artifacts are reacreated in the scratch dir with symlinks mostly for debugging purposes (to understand what artifact each hash corresponds to, and to follow what was built by the extractor).
The hash map mechanism that was already in use for reading swiftmodule
files on macOS is now in use also on Linux. The output replacing
mechanism has been also reworked so that the remapping is done directly in
the internal frontend options instead of painstakingly modifying input flags.
All outputs that are not
swiftmodulefiles are either removed or redirected to/dev/nullThe open file read redirection uses a global weak pointer instance to
maximize robustness in the face of possibly multi-threaded calls to open
happening while
mainis exiting. Possibly overkill, but better safethan sorry.