Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for relative 'file:' requirements in pip_install #367

Open
wants to merge 2 commits into
base: master
from

Conversation

@aaliddell
Copy link

@aaliddell aaliddell commented Oct 3, 2020

PR Checklist

Please check if your PR fulfills the following requirements:

  • Does not include precompiled binaries, eg. .par files. See CONTRIBUTING.md for info
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature (please, look at the "Scope of the project" section in the README.md file)
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Other... Please describe:

What is the current behavior?

Issue Number: closes #366

Relative file: requirements do not work due to pip wheel being run from the generated repository root.

What is the new behavior?

Relative file: URIs in requirements.txt are converted to absolute file:/// URIs based on the directory containing the requirements.txt file. This allows wheel files within the workspace to take part in dependency resolution, since the alternative whl_library rule is both non-recommended and does not attempt to fetch dependencies based on the wheel metadata.

Supports both file:some/path/to/wheel.whl and requirement @ file:some/path/to/wheel.whl. If a requirement is already absolute it will not be altered. For this to work, the requirements.txt file has to be copied into the generated repository, since we do not want to amend the source file in place.

This does not attempt to support local wheels with paths specified without file:, since these are equivalent but the bare requirement is harder to patch. Supporting that version consistently would require either writing a full parser for the requirements.txt format or depending on the internal implementation in pip._internal.

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

You may be able to deprecate whl_library rule as a result of this, if desired.

@aaliddell aaliddell requested review from brandjon and lberki as code owners Oct 3, 2020
@aaliddell
Copy link
Author

@aaliddell aaliddell commented Oct 3, 2020

I'm not sure how you'd like to test this, since we'd require having a .whl file (or other supported requirement type) checked into the repo, perhaps in the examples/pip_install folder. We could either choose to find a very small existing .whl file or create a tiny one ourselves. However, I can understand hesitation towards adding another binary file, since the .par files have been a pain to review in the past also.

Let me know what you'd like?

@pstradomski
Copy link
Collaborator

@pstradomski pstradomski commented Oct 4, 2020

@aaliddell
Copy link
Author

@aaliddell aaliddell commented Oct 4, 2020

Unfortunately I don't think that can work easily, since the wheel has to exist at the time the repo rules are run. The wheel files created in that folder won't be present until much later when the execution phase has completed.

Since they are technically separate workspaces, you could get it to work in CI by forcing the test order and copying the wheel at the right point, but it would not be a portable solution that would work for any contributors machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

3 participants
You can’t perform that action at this time.