Windows: Read ENV names and values as UTF-8 encoded Strings#3818
Conversation
Implements issue ruby#12650: https://bugs.ruby-lang.org/issues/12650 This also removes the special encoding for ENV['PATH'] and some complexity in the code that is unnecessary now.
|
Why just env encoding leaving the locale encoding? d9997d3 seems to change nothing. |
Not sure if I understand you right. If your question is about "What's wrong with keeping the current behavior of using locale encoding?", there are some reasons:
ENV["USERNAME"] = "Питер" # => "Питер"
ENV["USERNAME"] # => "?????" (not codable in CP850)
ENV['USERNAME'] = "Sören"
ENV['USERNAME'] # => "S\x94ren" (converted to CP850)
ENV['USERNAME'] =~ /Sör/ # => Encoding::CompatibilityError (incompatible encoding regexp match (UTF-8 regexp with CP850 string)) |
It changes nothing unless non-ASCII characters are used in |
|
By |
getenv() did use the expected codepage as an implicit parameter of the macro. This is mis-leading since include/ruby/win32.h has a different definition. Using the "cp" variable explicit (like the other function calls) makes it more readable and consistent.
They used to process and return strings with locale encoding, but since all ruby-internal spawn and environment functions use UTF-8, it makes sense to change the C-API equally.
|
@nobu You're absolutely right! After second reading it is just a cosmetic change. Still I find it a valuable change, since the implicit usage of |
|
Found that this |
|
Revived the macro by 3db21cf. |
|
That usage in dln_find.c is tricky - I wonder why CI didn't fail. But anyway, let me know if I should rebase the PR to remove the second commit and to integrate the latest patches that fix the MJIT failures in CI! |
|
The failures seem caused by MJIT, unrelated to this PR. |
3 participants
This implements redmine issue 12650, which was postponed to ruby-3.0, but isn't solved so far.
I splitted the PR into 3 commits, which are not necessary related. Please tell me, if I should open 3 separate PRs.
0c6c879: Read ENV names and values as UTF-8 encoded Strings
Implements issue #12650: https://bugs.ruby-lang.org/issues/12650 This also removes the special encoding for
ENV['PATH']and some complexity in the code that is unnecessary now.9532666: Improve readablity of
getenv()encodinggetenv()did use the expected codepage as an implicit parameter of the macro. This is mis-leading sinceinclude/ruby/win32.hhas a different definition. Using the "cp" variable explicit (like the other function calls) makes it more readable and consistent.26ffdf0: Change external C-API macros
getenv()andexecv()to use UTF-8They used to process and return strings with locale encoding, but since all ruby-internal spawn and environment functions use UTF-8, it makes sense to change the C-API equally.
I'm uncertain if this commit is desired, since it changes the C-API, so that getenv() and execv() might not be fully compatible to getenv() from stdlib.h or execv(). On the other hand it seems more handy for work with UTF-8 in ruby extensions. Let me know, if I should remove this commit.
Please also note, that there's PR #2877 which is also related to UTF-8 on Windows and was postponed to ruby-3.0.