Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of python's caching #266

Merged
merged 13 commits into from Nov 17, 2021

Conversation

dmitry-shibanov
Copy link
Contributor

@dmitry-shibanov dmitry-shibanov commented Nov 10, 2021

In scope of this pull request we add possibility to cache python dependencies. We added cache input parameter to the action. For now, input will accept the following values:

  • pip - enable caching for pip dependencies
  • pipenv - enable caching for pipenv dependencies

ADR: #247
Related issue: #226

Description

Action will try to search requirements.txt for pip or Pipfile.lock for pipenv files in the repository and throw error if no one is found. The hash of found files will be used as part of cache key (the same approach like actions/cache recommends). The following key cache will be used:

  • For pip: setup-python-${{ runner.os }}-pip-${{ hashFiles('') }}
  • For pipenv: setup-python-${{ runner.os }}-python-${{resolved-python-version}}-pipenv-${{ hashFiles('') }}

Action will cache:

  • Pip (global cache directory retrieved via pip cache dir)
  • Pipenv (virtualenv directories $HOME/.virtualenvs for Windows and $HOME/.local/share/virtualenvs for Linux, macOS)

Besides we add cache-dependency-path input. This input will accept an array or regex of dependency files. The field will accept a path (relative to the repository root) to dependency files. If the provided path contains wildcards, the action will search all matching files and calculate a common hash like the ${{ hashFiles('**/requirements-dev.txt') }} YAML construction does.

Example of yml

  • For pip
 uses: actions/setup-python@v2
  with:
    python-version: '3.9'
    cache: 'pip'
  • For pipenv:
 uses: actions/setup-python@v2
  with:
    python-version: '3.9'
    cache: 'pipenv'

Note:

  • It's not breaking change because it requires additional field and nothing changes by default.
  • License tests are falling. But we have changes to fix it. We've decided to stay with implementation, because approximately 60 licenses were changed. When changes are reviewed, we will add licenses to the existing pull request.

What's done:

  • - Implementation of python's caching
  • - Adding tests for python's caching
  • - Adding documentation
  • - Update licenses

Copy link

@yaananth yaananth left a comment

just minor clean up comments, didn't look closely at core logics yet. I will defer it to someone else or I can come back and take a peek something this week.

Thanks!

.github/workflows/e2e-cache.yml Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
__tests__/cache-restore.test.ts Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
action.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-cache.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-cache.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-cache.yml Outdated Show resolved Hide resolved
src/setup-python.ts Outdated Show resolved Hide resolved
@merwok
Copy link

@merwok merwok commented Nov 12, 2021

Tox is another wildly popular tool that manages its own virtual envs and installs dependencies.
Could there be an option to cache .tox directory too? (not exclusive with pip caching but combinable)

src/setup-python.ts Outdated Show resolved Hide resolved
@konradpabjan
Copy link
Collaborator

@konradpabjan konradpabjan commented Nov 15, 2021

Tox is another wildly popular tool that manages its own virtual envs and installs dependencies.
Could there be an option to cache .tox directory too? (not exclusive with pip caching but combinable)

@merwok In a future PR this is something that can definitely be added. I'd be happy to review any contributor PR in the future that adds this support. For the scope of this PR though we're just focusing first on pip and pipenv

@konradpabjan
Copy link
Collaborator

@konradpabjan konradpabjan commented Nov 15, 2021

@dmitry-shibanov This is looking great! Lets fix the failing Licensed check and there is that cacheDepencies mispelling and IMO we're good to go 🚀

Copy link

@yaananth yaananth left a comment

some questions/suggestions to get more clarity, but 👍

src/cache-distributions/pipenv-cache.ts Show resolved Hide resolved
src/setup-python.ts Show resolved Hide resolved
@MaksimZhukov MaksimZhukov merged commit 280924f into actions:main Nov 17, 2021
129 of 130 checks passed
@hugovk
Copy link
Contributor

@hugovk hugovk commented Nov 17, 2021

This is great, thank you! And for the new release: https://github.com/actions/setup-python/releases/tag/v2.3.0

Please could you also update the v2 tag to point to v2.3.0?

@dmitry-shibanov
Copy link
Contributor Author

@dmitry-shibanov dmitry-shibanov commented Nov 17, 2021

Hello @hugovk. Thank you. I think we'll update v2 tag tomorrow. We want to proceed with some extra validation.

@dmitry-shibanov
Copy link
Contributor Author

@dmitry-shibanov dmitry-shibanov commented Nov 18, 2021

Hello @hugovk. We've updated the v2 tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants