Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Require Jinja2 3.0.0 #75881

Merged
merged 18 commits into from Oct 20, 2021
Merged

Require Jinja2 3.0.0 #75881

merged 18 commits into from Oct 20, 2021

Conversation

mkrizek
Copy link
Contributor

@mkrizek mkrizek commented Oct 4, 2021

SUMMARY

ci_complete

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

many

@ansibot

This comment has been hidden.

Sign in to view
@sivel sivel added this to Approved: Prioritized work that blocks release. All items here must be assigned in ansible-core 2.13 Oct 4, 2021
@mattclay
Copy link
Member

@mattclay mattclay commented Oct 6, 2021

@mkrizek ansible-test should be updated to not install jinja2 using OS packages on RHEL by removing this line:

ansible/test/lib/ansible_test/_util/target/setup/bootstrap.sh

Line 244 in 2f531d7

${py_pkg_prefix}-jinja2

The FreeBSD setup should also be updated to remove jinja2 OS package references in this section:

ansible/test/lib/ansible_test/_util/target/setup/bootstrap.sh

Lines 153 to 163 in 2f531d7

jinja2_pkg="py${python_package_version}-Jinja2"
cryptography_pkg="py${python_package_version}-cryptography"
else
jinja2_pkg=""
cryptography_pkg=""
fi
packages="
${packages}
libyaml
${jinja2_pkg}

In both cases I'd include a comment above those that states something like:

# Jinja2 is not installed with an OS package since the provided version is too old.
# Instead, ansible-test will install it using pip.

@mattclay
Copy link
Member

@mattclay mattclay commented Oct 6, 2021

Now that the min and max functions have been removed from mathstuff the unit tests also need to be removed:

ansible/test/units/plugins/filter/test_mathstuff.py

Lines 65 to 82 in 9a4346b

class TestMin:
def test_min(self):
assert ms.min(env, (1, 2)) == 1
assert ms.min(env, (2, 1)) == 1
assert ms.min(env, ('p', 'a', 'w', 'b', 'p')) == 'a'
assert ms.min(env, ({'key': 'a'}, {'key': 'b'}, {'key': 'c'}), attribute='key') == {'key': 'a'}
assert ms.min(env, ({'key': 1}, {'key': 2}, {'key': 3}), attribute='key') == {'key': 1}
assert ms.min(env, ('a', 'A', 'b', 'B'), case_sensitive=True) == 'A'
class TestMax:
def test_max(self):
assert ms.max(env, (1, 2)) == 2
assert ms.max(env, (2, 1)) == 2
assert ms.max(env, ('p', 'a', 'w', 'b', 'p')) == 'w'
assert ms.max(env, ({'key': 'a'}, {'key': 'b'}, {'key': 'c'}), attribute='key') == {'key': 'c'}
assert ms.max(env, ({'key': 1}, {'key': 2}, {'key': 3}), attribute='key') == {'key': 3}
assert ms.max(env, ('a', 'A', 'b', 'B'), case_sensitive=True) == 'b'

@mattclay
Copy link
Member

@mattclay mattclay commented Oct 6, 2021

With the changes in this PR we should be able to clean up the integration tests quite a bit:

  • We should no longer need to install, uninstall, upgrade or downgrade jinja2 in integration tests.
  • We should no longer need to check the installed jinja2 version.

Some of the current CI failures will be resolved by making these changes.

As part of these changes, some of the integration tests that are script based may be able to be converted to role based.

@ansibot

This comment has been hidden.

Sign in to view
@ansibot ansibot added needs_ci and removed needs_ci labels Oct 8, 2021
@mkrizek mkrizek marked this pull request as ready for review Oct 11, 2021
@ansibot ansibot added core_review and removed WIP labels Oct 11, 2021
sivel
sivel approved these changes Oct 20, 2021
View changes
@sivel
Copy link
Member

@sivel sivel commented Oct 20, 2021

/azp run

@azure-pipelines
Copy link

@azure-pipelines azure-pipelines bot commented Oct 20, 2021

Azure Pipelines successfully started running 1 pipeline(s).

@sivel sivel merged commit 7621784 into ansible:devel Oct 20, 2021
124 checks passed
ansible-core 2.13 automation moved this from Approved: Prioritized work that blocks release. All items here must be assigned to Merged Oct 20, 2021
@mkrizek mkrizek deleted the jinja2-3 branch Oct 21, 2021
@mkrizek mkrizek mentioned this pull request Oct 21, 2021
Closed
@sivel sivel moved this from Merged to Done in ansible-core 2.13 Oct 21, 2021
@AlanCoding
Copy link
Member

@AlanCoding AlanCoding commented Oct 25, 2021

I just want to paste this here:

Traceback (most recent call last):
  File "/home/alancoding/repos/ansible/lib/ansible/template/__init__.py", line 559, in __getitem__
    raise TemplateSyntaxError(to_native(e), 0)
jinja2.exceptions.TemplateSyntaxError: cannot import name 'pass_environment' from 'jinja2.filters' (/home/alancoding/repos/awx/env/lib64/python3.9/site-packages/jinja2/filters.py)
  line 0
exception during Jinja2 environment setup: Traceback (most recent call last):
  File "/home/alancoding/repos/ansible/lib/ansible/template/__init__.py", line 557, in __getitem__
    plugin_impl = self._pluginloader.get(module_name)
  File "/home/alancoding/repos/ansible/lib/ansible/plugins/loader.py", line 1018, in get
    return super(Jinja2Loader, self).get(name, *args, **kwargs)
  File "/home/alancoding/repos/ansible/lib/ansible/plugins/loader.py", line 810, in get
    return self.get_with_context(name, *args, **kwargs).object
  File "/home/alancoding/repos/ansible/lib/ansible/plugins/loader.py", line 830, in get_with_context
    self._module_cache[path] = self._load_module_source(name, path)
  File "/home/alancoding/repos/ansible/lib/ansible/plugins/loader.py", line 794, in _load_module_source
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/alancoding/repos/ansible/lib/ansible/plugins/filter/core.py", line 25, in <module>
    from jinja2.filters import pass_environment
ImportError: cannot import name 'pass_environment' from 'jinja2.filters' (/home/alancoding/repos/awx/env/lib64/python3.9/site-packages/jinja2/filters.py)

Because, given time, a lot of people are probably going to hit this. Resolving it is a simple matter of upgrading jinja2

@mkrizek mkrizek mentioned this pull request Oct 26, 2021
Merged
@ansible ansible locked and limited conversation to collaborators Nov 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bcoca bcoca

@sivel sivel

Assignees
No one assigned
Labels
affects_2.13 core_review feature support:community support:core test
Projects
ansible-core 2.13
  
Done
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

None yet
6 participants
@mkrizek @ansibot @mattclay @sivel @AlanCoding @bcoca