Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import completion crawls too many files #845

Open
thomasballinger opened this issue Aug 23, 2020 · 2 comments
Open

Import completion crawls too many files #845

thomasballinger opened this issue Aug 23, 2020 · 2 comments
Labels
Milestone

Comments

@thomasballinger
Copy link
Member

@thomasballinger thomasballinger commented Aug 23, 2020

@pyang30 as you mentioned in #806 I'm seeing bpython access a ton of files it shouldn't need to in order to initialize the import completion set. Like for you, it spends most of its time in node_modules folders.

There's something very broken here, we're crawling way too much.

@thomasballinger
Copy link
Member Author

@thomasballinger thomasballinger commented Aug 23, 2020

This isn't broken! Our import completion correctly traverses directories which do not contain an __init__.py, since #792, because empty __init__.py files are no longer required for namespace packages.

  • Maybe we shouldn't crawl these at all. Apparently unittest doesn't crawl them, a sort of similar problem. At some point in the past, IPython skipped these too.

  • Maybe we should use different crawling logic for the current directory that bpython is run from than we do for other locations on sys.path: namespace packages in other locations will still be picked up, but running bpython from a filesystem path which contains a massive node_modules directory or similar won't chew up CPU. Users who manually add e.g. their home directory to sys.path would still see this issue.

@sebastinas
Copy link
Contributor

@sebastinas sebastinas commented Aug 23, 2020

We could implement a (user-configurable) skip list which is checked before traversing into directories. We could make it default to node_modules, .git, and maybe other well known directories which won't contain Python modules.

@sebastinas sebastinas added this to the release-0.20 milestone Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.