Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3.12 Support #33063

Closed
Tracked by #1862
leslieluyu opened this issue May 10, 2023 · 4 comments
Closed
Tracked by #1862

Python 3.12 Support #33063

leslieluyu opened this issue May 10, 2023 · 4 comments
Assignees
@gnossen @XuanWang-Amos
Labels
kind/enhancement lang/Python priority/P2

Comments

@leslieluyu
Copy link

What version of gRPC and what language are you using?

Python 3.12.0a7
grpcio 1.53.0, 1.54.0

What operating system (Linux, Windows,...) and version?

in a container: Debian GNU/Linux 11 (bullseye)

What runtime / compiler are you using (e.g. python version or version of gcc)

python 3.12.0a7

What did you do?

pip install grpcio==1.54.0

What did you expect to see?

A successful install.

What did you see instead?

can't find the related wheel, so begin to build from the downloaded source code , but failed:
part of the error log:

      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -D_WIN32_WINNT=1536 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"Python\" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"1.54.0\" -DGPR_BACKWARDS_COMPATIBILITY_MODE=1 -DHAVE_CONFIG_H=1 -DGRPC_ENABLE_FORK_SUPPORT=1 "-DPyMODINIT_FUNC=extern \"C\" __attribute__((visibility (\"default\"))) PyObject*" -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -Isrc/python/grpcio -Iinclude -I. -Ithird_party/abseil-cpp -Ithird_party/address_sorting/include -Ithird_party/cares/cares/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/cares/config_linux -Ithird_party/re2 -Ithird_party/boringssl-with-bazel/src/include -Ithird_party/upb -Isrc/core/ext/upb-generated -Isrc/core/ext/upbdefs-generated -Ithird_party/xxhash -Ithird_party/zlib -I/usr/local/include/python3.12 -c src/core/ext/filters/client_channel/client_channel_channelz.cc -o python_build/temp.linux-x86_64-cpython-312/src/core/ext/filters/client_channel/client_channel_channelz.o -std=c++14 -fvisibility=hidden -fno-wrapv -fno-exceptions -pthread
      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -D_WIN32_WINNT=1536 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"Python\" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"1.54.0\" -DGPR_BACKWARDS_COMPATIBILITY_MODE=1 -DHAVE_CONFIG_H=1 -DGRPC_ENABLE_FORK_SUPPORT=1 "-DPyMODINIT_FUNC=extern \"C\" __attribute__((visibility (\"default\"))) PyObject*" -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -Isrc/python/grpcio -Iinclude -I. -Ithird_party/abseil-cpp -Ithird_party/address_sorting/include -Ithird_party/cares/cares/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/cares/config_linux -Ithird_party/re2 -Ithird_party/boringssl-with-bazel/src/include -Ithird_party/upb -Isrc/core/ext/upb-generated -Isrc/core/ext/upbdefs-generated -Ithird_party/xxhash -Ithird_party/zlib -I/usr/local/include/python3.12 -c src/core/ext/filters/client_channel/retry_service_config.cc -o python_build/temp.linux-x86_64-cpython-312/src/core/ext/filters/client_channel/retry_service_config.o -std=c++14 -fvisibility=hidden -fno-wrapv -fno-exceptions -pthread
      creating None
      creating None/tmp
      creating None/tmp/tmpt9aklqte
      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/usr/local/include/python3.12 -c /tmp/tmpt9aklqte/a.c -o None/tmp/tmpt9aklqte/a.o
      Traceback (most recent call last):
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
          _classic_spawn(self, command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/commands.py", line 281, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/usr/local/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 367, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
          raise self._value
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
                          ^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
                 ^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/setup.py", line 537, in <module>
          setuptools.setup(
        File "/usr/local/lib/python3.12/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/usr/local/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
          super().run_command(command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/usr/local/lib/python3.12/site-packages/wheel/bdist_wheel.py", line 343, in run
          self.run_command("build")
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
          super().run_command(command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/dist.py", line 1244, in run_command
          super().run_command(command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/usr/local/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 84, in run
          _build_ext.run(self)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/commands.py", line 285, in build_extensions
          raise CommandError(
      commands.CommandError: Failed `build_ext` step:
      Traceback (most recent call last):
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
          _classic_spawn(self, command)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/commands.py", line 281, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/usr/local/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 367, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
          raise self._value
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
                          ^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
                 ^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-2ll9ksbb/grpcio_1703c0a9ac2440098f1de906670dcc7d/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/usr/local/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for grpcio
  Running setup.py clean for grpcio
Failed to build grpcio
ERROR: Could not build wheels for grpcio, which is required to install pyproject.toml-based projects

Anything else we should know about your project / environment?

Couldn't find the grpcio wheels for python3.12 in here: https://pypi.org/project/grpcio/#files
@XuanWang-Amos
Copy link
Contributor

XuanWang-Amos commented May 10, 2023 •
edited
Loading

We're still working on this, we'll have official support before Python3.12 official release.

@XuanWang-Amos XuanWang-Amos changed the title Can't install or build grpcio for python3.12 Python 3.12 Support May 10, 2023
@leslieluyu
Copy link
Author

Thanks for the quick reply! Looking forward to the official release. The reason I need python3.12 is that I want to try the new feature of support for the Linux perf profiler

@miketheman miketheman mentioned this issue May 25, 2023
Closed
10 tasks
This was referenced Jun 20, 2023
Merged
Merged
gnossen pushed a commit that referenced this issue Jun 21, 2023
@musicinmybrain
[Test] Use ssl.SSLContext.wrap_socket, not ssl.wrap_socket (#33492)
011e116 
In the HTTP(S) test server in the core tests, use
`ssl.SSLContext.wrap_socket`, not `ssl.wrap_socket`. The latter emits a
`DeprecationWarning` since Python 3.10 and is [removed in Python
3.12](python/cpython#94199).

This fixes the core tests (but not necessarily the `grpcio` tests) for
Python 3.12.

This is relevant to #33063.
@nijel nijel mentioned this issue Jul 14, 2023
Merged
5 tasks
@cdce8p cdce8p mentioned this issue Jul 19, 2023
Closed
32 tasks
@JacobCoffee JacobCoffee mentioned this issue Jun 22, 2023
Closed
8 tasks
gnossen pushed a commit that referenced this issue Sep 6, 2023
@musicinmybrain
[Test] Do not use importlib find_module API, removed in Python 3.12 (#…
efc3843 
…33506)

This API was [removed in Python
3.12](python/cpython#98040).

Fixes Python 3.12 support in `grpcio` tests.

This is relevant to #33063.

See also #33492.

----

I have actually only tested this in a form backported to grpc 1.48.4,
and I am not able to test the change to `bazel/_gevent_test_main.py`
directly. However, the backported form allows me to build grpc 1.48.4
for Fedora Rawhide with Python 3.12, and I believe the version in this
PR to be correct—especially, if CI passes for Python 3.11, I believe
this part of the test code will continue to work in Python 3.12.
@raulcd raulcd mentioned this issue Sep 27, 2023
Merged
@miketheman
Copy link

Resolved by #34398 ?

@gnossen
Copy link
Contributor

gnossen commented Sep 29, 2023

Yup. Resolved by #34398

@gnossen gnossen closed this as completed Sep 29, 2023
@raulcd raulcd mentioned this issue Oct 4, 2023
Closed
@defreng defreng mentioned this issue Oct 16, 2023
Closed
antoninbas added a commit to antoninbas/p4c that referenced this issue Nov 10, 2023
@antoninbas
Update gRPC python version in install_fedora_deps.sh
dcbe197 
The fedora:latest container now comes with Python 3.12. Older gRPC
versions do not support this Python version. See
grpc/grpc#33063.

An alternative would be to pin the fedora container version, but then
someone has to remember to update it periodically...

Signed-off-by: Antonin Bas <antonin.bas@gmail.com>
@antoninbas antoninbas mentioned this issue Nov 10, 2023
Merged
antoninbas added a commit to antoninbas/p4c that referenced this issue Nov 10, 2023
@antoninbas
Update gRPC python version in install_fedora_deps.sh
15fafc1 
The fedora:latest container now comes with Python 3.12. Older gRPC
versions do not support this Python version. See
grpc/grpc#33063.

An alternative would be to pin the fedora container version, but then
someone has to remember to update it periodically...

Signed-off-by: Antonin Bas <antonin.bas@gmail.com>
antoninbas added a commit to antoninbas/p4c that referenced this issue Nov 10, 2023
@antoninbas
Update gRPC python version in install_fedora_deps.sh
32b026b 
The fedora:latest container now comes with Python 3.12. Older gRPC
versions do not support this Python version. See
grpc/grpc#33063.

An alternative would be to pin the fedora container version, but then
someone has to remember to update it periodically...

Signed-off-by: Antonin Bas <antonin.bas@gmail.com>
antoninbas added a commit to antoninbas/p4c that referenced this issue Nov 10, 2023
@antoninbas
Update gRPC python version in install_fedora_deps.sh
e6f2a26 
The fedora:latest container now comes with Python 3.12. Older gRPC
versions do not support this Python version. See
grpc/grpc#33063.

An alternative would be to pin the fedora container version, but then
someone has to remember to update it periodically...

Signed-off-by: Antonin Bas <antonin.bas@gmail.com>
antoninbas added a commit to p4lang/p4c that referenced this issue Nov 10, 2023
@antoninbas
Fix build for latest Fedora version (#4241)
fe7b267 
* Update gRPC python version in install_fedora_deps.sh

The fedora:latest container now comes with Python 3.12. Older gRPC
versions do not support this Python version. See
grpc/grpc#33063.

An alternative would be to pin the fedora container version, but then
someone has to remember to update it periodically...

* Update scapy from 2.4.5 to 2.5.0

Required for more recent versions of Python3

* Fix usage of tempfile.mkdir

Changed in Python version 3.12: mkdtemp() now always returns an absolute path,
even if dir is relative.

Signed-off-by: Antonin Bas <antonin.bas@gmail.com>
@jamesbuddrige jamesbuddrige mentioned this issue Nov 19, 2023
Closed
@sehoffmann sehoffmann mentioned this issue Apr 1, 2024
Closed
16 tasks
@dirkpetersen dirkpetersen mentioned this issue May 1, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gnossen gnossen

@XuanWang-Amos XuanWang-Amos

Labels
kind/enhancement lang/Python priority/P2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@miketheman @gnossen @leslieluyu @XuanWang-Amos