Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) #83386

Open
brianquinlan opened this issue Jan 3, 2020 · 2 comments
Open

multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) #83386

brianquinlan opened this issue Jan 3, 2020 · 2 comments
Assignees
@brianquinlan
Labels
3.9 type-bug

Comments

@brianquinlan
Copy link
Contributor

@brianquinlan brianquinlan commented Jan 3, 2020

BPO 39205
Nosy @brianquinlan
PRs
  • #18221
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/brianquinlan'
closed_at = None
created_at = <Date 2020-01-03.21:26:24.394>
labels = ['type-bug', '3.9']
title = 'multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False)'
updated_at = <Date 2020-01-28.01:40:13.223>
user = 'https://github.com/brianquinlan'

    bugs.python.org fields:

    activity = <Date 2020-01-28.01:40:13.223>
actor = 'vstinner'
assignee = 'bquinlan'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-01-03.21:26:24.394>
creator = 'bquinlan'
dependencies = []
files = []
hgrepos = []
issue_num = 39205
keywords = ['patch']
message_count = 2.0
messages = ['359257', '360828']
nosy_count = 1.0
nosy_names = ['bquinlan']
pr_nums = ['18221']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39205'
versions = ['Python 3.9']
    @brianquinlan
    Copy link
    Contributor Author

    @brianquinlan brianquinlan commented Jan 3, 2020 

    from concurrent.futures import ProcessPoolExecutor
import time

t = ProcessPoolExecutor(max_workers=3)
t.map(time.sleep, [1,2,3])
t.shutdown(wait=False)

    Results in this exception and then a hang (i.e. Python doesn't terminate):

    Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/usr/local/google/home/bquinlan/cpython/Lib/threading.py", line 944, in _bootstrap_inner
    self.run()
  File "/usr/local/google/home/bquinlan/cpython/Lib/threading.py", line 882, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/google/home/bquinlan/cpython/Lib/concurrent/futures/process.py", line 352, in _queue_management_worker
    _add_call_item_to_queue(pending_work_items,
  File "/usr/local/google/home/bquinlan/cpython/Lib/concurrent/futures/process.py", line 280, in _add_call_item_to_queue
    call_queue.put(_CallItem(work_id,
  File "/usr/local/google/home/bquinlan/cpython/Lib/multiprocessing/queues.py", line 82, in put
    raise ValueError(f"Queue {self!r} is closed")
ValueError: Queue <concurrent.futures.process._SafeQueue object at 0x7f371c4ae7f0> is closed

    @brianquinlan brianquinlan self-assigned this Jan 3, 2020
    @brianquinlan brianquinlan added type-bug 3.9 labels Jan 3, 2020
    @brianquinlan brianquinlan self-assigned this Jan 3, 2020
    @brianquinlan brianquinlan added the type-bug label Jan 3, 2020
    @brianquinlan
    Copy link
    Contributor Author

    @brianquinlan brianquinlan commented Jan 28, 2020

    New changeset 884eb89 by Brian Quinlan in branch 'master':
    bpo-39205: Tests that highlight a hang on ProcessPoolExecutor shutdown (bpo-18221)
    884eb89

    @vstinner vstinner changed the title Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) Jan 28, 2020
    @vstinner vstinner changed the title Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False) Jan 28, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @brianquinlan brianquinlan

    Labels
    3.9 type-bug
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @brianquinlan